8 Replies Latest reply on Feb 3, 2020 10:04 PM by Nico Heinze

    Unable to bring up Informatica services in 9.6.1

    Sudharsan Kannan New Member

      Unable to bring up informatica services in 9.6.1 getting below error in mentioned files.

      Note:

      am trying to bring up services in DR which was sync up with PROD in earlier.

       

      $INFA_HOME/isp/bin/infasetup_jsf.log:

      WARN  [main][com.informatica.componentfw.ComponentFramework] [COMPONENT_0005] Failed to load component 'PortalsProvider'.

      org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean named 'PortalsProvider' is defined

              at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeanDefinition(DefaultListableBeanFactory.java:391)

              at org.springframework.beans.factory.support.AbstractBeanFactory.getMergedLocalBeanDefinition(AbstractBeanFactory.java:999)

              at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:233)

              at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:170)

              at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:883)

              at com.informatica.componentfw.ComponentFramework.getComponent(ComponentFramework.java:90)

              at com.informatica.project.repository.client.impl.ProjectRepoSessionFactoryImpl.<clinit>(ProjectRepoSessionFactoryImpl.java:170)

              at com.informatica.project.repository.deployer.embedded.EmbeddedConnectionFactory.connect(EmbeddedConnectionFactory.java:246)

              at com.informatica.isp.domainrepositoryservice.impl.PRSDomainRepositoryServiceImpl.initImpl(PRSDomainRepositoryServiceImpl.java:200)

              at com.informatica.isp.domainrepositoryservice.impl.PRSDomainRepositoryServiceImpl.init(PRSDomainRepositoryServiceImpl.java:526)

              at com.informatica.isp.infasetup.common.InfasetupHelper.initDomainRepository(InfasetupHelper.java:114)

              at com.informatica.isp.infasetup.impl.ConfigDomainPluginImpl.migrateDomainMetadata(ConfigDomainPluginImpl.java:2163)

              at com.informatica.isp.infasetup.impl.ConfigDomainPluginImpl.migrateEncryptionKey(ConfigDomainPluginImpl.java:2131)

              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

              at java.lang.reflect.Method.invoke(Method.java:606)

              at com.informatica.pcsf.cmdframework.PluginCommand.run(PluginCommand.java:46)

              at com.informatica.pcsf.cmdframework.Infasetup.execute(Infasetup.java:224)

              at com.informatica.pcsf.cmdframework.Infasetup.main(Infasetup.java:87)

       

      $INFA_HOME/tomcat/logs/catalina.out:

       

      Jan 27, 2020 4:40:35 AM com.informatica.repository.svcfw.SvcLogService log

      SEVERE: The Repository Service operation failed. ['[RSVCSHARED_00106] The repository content for Repository Service ['ISP-Repository'] requires an upgrade using new encryption key.']

      com.informatica.repository.shared.exceptions.RepositoryMigrationException: [RSVCSHARED_00106] The repository content for Repository Service ['ISP-Repository'] requires an upgrade using new encryption key.

              at com.informatica.project.repository.service.infrastructure.interceptors.PrerequisiteCheckRequestInterceptor.beforeRequestHandling(PrerequisiteCheckRequestInterceptor.java:69)

              at com.informatica.repository.base.infrastructure.service.impl.ServiceInterceptorManagerImpl.executeRequestInterceptors(ServiceInterceptorManagerImpl.java:422)

              at com.informatica.repository.base.infrastructure.service.impl.ServiceInterceptorManagerImpl.executeBeforeRequestInterceptors(ServiceInterceptorManagerImpl.java:309)

              at com.informatica.project.repository.service.impl.ProjectRepoServiceFacadeProxy.beforeMethodDispatching(ProjectRepoServiceFacadeProxy.java:659)

              at com.informatica.project.repository.service.impl.ProjectRepoServiceFacadeProxy.invoke(ProjectRepoServiceFacadeProxy.java:176)

              at com.sun.proxy.$Proxy6.lookupProjectIdByName(Unknown Source)

              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

              at java.lang.reflect.Method.invoke(Method.java:606)

              at com.informatica.project.repository.client.impl.SessionSvcFacadeWrapper.resilientMethodInvocation(SessionSvcFacadeWrapper.java:232)

              at com.informatica.project.repository.client.impl.SessionSvcFacadeWrapper.invoke(SessionSvcFacadeWrapper.java:160)

              at com.sun.proxy.$Proxy11.lookupProjectIdByName(Unknown Source)

              at com.informatica.project.repository.client.impl.ProjectRepoAdminSessionImpl.lookupProjectIdByName(ProjectRepoAdminSessionImpl.java:226)

              at com.informatica.project.repository.client.impl.ProjectRepoAdminSessionImpl.lookupProjectIdByName(ProjectRepoAdminSessionImpl.java:191)

              at com.informatica.isp.domainrepositoryservice.impl.PRSDomainRepositoryServiceImpl.initImpl(PRSDomainRepositoryServiceImpl.java:212)

              at com.informatica.isp.domainrepositoryservice.impl.PRSDomainRepositoryServiceImpl.init(PRSDomainRepositoryServiceImpl.java:526)

              at com.informatica.isp.tools.ispstarter.JSFStarter.init(JSFStarter.java:189)

              at com.informatica.isp.tools.ispwebstarter.ISPWebStarterServlet.init(ISPWebStarterServlet.java:24)

              at javax.servlet.GenericServlet.init(GenericServlet.java:158)

              at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1284)

              at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1197)

              at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1087)

              at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5210)

              at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5493)

              at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)

              at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:901)

              at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:877)

              at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:632)

              at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:1073)

              at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1857)

              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)

              at java.util.concurrent.FutureTask.run(FutureTask.java:262)

              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

              at java.lang.Thread.run(Thread.java:744)

       

       

      Cannot start node because of the following error: [[RSVCSHARED_00106] The repository content for Repository Service ['ISP-Repository'] requires an upgrade using new encryption key.].

      Jan 27, 2020 4:40:35 AM org.apache.catalina.core.StandardContext loadOnStartup

      SEVERE: Servlet /coreservices threw load() exception

      javax.servlet.ServletException: Cannot start node because of the following error: [[RSVCSHARED_00106] The repository content for Repository Service ['ISP-Repository'] requires an upgrade using new encryption key.].

              at com.informatica.isp.tools.ispwebstarter.ISPWebStarterServlet.init(ISPWebStarterServlet.java:42)

              at javax.servlet.GenericServlet.init(GenericServlet.java:158)

              at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1284)

              at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1197)

              at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1087)

              at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5210)

              at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5493)

              at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)

              at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:901)

              at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:877)

              at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:632)

              at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:1073)

              at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1857)

              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)

              at java.util.concurrent.FutureTask.run(FutureTask.java:262)

              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

              at java.lang.Thread.run(Thread.java:744)

       

      will share more details, if any required. Thanks in advance.

        • 1. Re: Unable to bring up Informatica services in 9.6.1
          Nico Heinze Guru

          How did you sync the Disaster Recovery machine with the PRO machine?

          There are many ways to do it the wrong way, that's why I'm asking for the exact procedure you followed.

           

          Regards,

          Nico

          • 2. Re: Unable to bring up Informatica services in 9.6.1
            Sudharsan Kannan New Member

            Thanks Nico.

             

            my senior folks were just created the DR  servers and installed everything and just make sync with production databases and they moved to another project. We never involved the DR activities and trying to make sure everything fine in the DR environment.

             

            SYNC-UP process:

            Production database is PROD and DR database is DR and both the DB's are in sync. So below tables in DR having Production details such as host_name, node_name and repository_name like that.

             

            PO_DOMAINOBJ

            ISP_MASTER_ELECTION

            ISP_MASTER_ELECT_LOCK

             

            Note:

            There are so many configuration tables in DB and i just mentioned here for quick reference.

            • 3. Re: Unable to bring up Informatica services in 9.6.1
              Nico Heinze Guru

              You have one small chance:

              Switch the IP address and host name of the DR machine to the former production machine.

              Bring it up using the old PROD domain repository.

              Take a backup of the PROD repository (.mrep file, if memory serves me right).

              Clear the current DR database. It's useless anyway.

              Restore that .mrep file in the DR database.

              Switch the DR machine back to the DR IP address and host name.

              Try to bring up the the DR domain.

               

              A very small chance, and I've never tried that.

              May I suggest that you raise a Service Request with Informatica Global Customer Support (GCS). These guys may be able to help you. If not them, then probably no one can help you.

               

              Sorry for the bad news, but taking a DB backup of any Informatica repository for use on a different machine (be it a different DB server or a different INFA server) is one of the most stupid ideas that anyone could implement. Sorry for the harsh words, that's nothing but the naked truth.

               

              Regards,

              Nico

              • 4. Re: Unable to bring up Informatica services in 9.6.1
                Sudharsan Kannan New Member

                Thanks for your suggestion Nico.

                 

                Before applying it, PROD is live now and Domain database is down in DR, but both the hosts are up now, means Production host and DR host. so we cant switch the IP address and host name of the DR machine to the production machine.

                 

                 

                I have taken the backup of domain and repository as well from Production and copied it to DR server.

                Next i tried to restore the production domain into DR server using restoreDomain command, but throws below error.

                 

                 

                [ICMD_10033] Command [RestoreDomain] failed with error [[INFASETUP_10039] Cannot restore the domain because of the following error: [PRSVCSHARED_00006] Repository ['ISP-Repository'] contains content. Before creating content for a repository, verify that no content exists..]

                 

                 

                Few KB pages suggested to do the deletion of contents in repository, use pmrep command.

                So i started from 'pmrep connect' command, it throws 'Failed to execute connect'.

                 

                pmrep connect -r rep -d Domain

                 

                What is the possible way to find out the problem with pmrep connect command?

                • 5. Re: Unable to bring up Informatica services in 9.6.1
                  Nico Heinze Guru

                  pmrep is only for PowerCenter repositories.

                  For the domain repository and for all sorts of platform repositories you need infacmd (resp. infacmd.sh).

                   

                  Before you can restore a repository from a backup file, the respective database (resp. schema) into which you want to restore has to be empty. No tables, no views, nothing. You will probably have to clean up the former mess manually using some DB tool, because I fear that you cannot remove the contents from the Administrator tool (after all the system seems to be slightly corrupt).

                   

                  Regards,

                  Nico

                  • 6. Re: Unable to bring up Informatica services in 9.6.1
                    Sudharsan Kannan New Member

                    After executing

                    infasetup_jsf.log Error:

                    2020-02-02 07:27:48,342 ERROR [main][org.hibernate.event.def.AbstractFlushingEventListener] Could not synchronize database state with session

                    org.hibernate.exception.GenericJDBCException: could not insert: [com.informatica.repository.persistence.databasemodel.impl.UpgradeLogEntryImpl]

                            at org.hibernate.exception.SQLStateConverter.handledNonSpecificException(SQLStateConverter.java:126)

                            at org.hibernate.exception.SQLStateConverter.convert(SQLStateConverter.java:114)

                            at org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:66)

                            at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:2285)

                            at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:2678)

                            at org.hibernate.action.EntityInsertAction.execute(EntityInsertAction.java:79)

                            at org.hibernate.engine.ActionQueue.execute(ActionQueue.java:279)

                            at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:263)

                            at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:167)

                            at org.hibernate.event.def.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:321)

                            at org.hibernate.event.def.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:50)

                            at org.hibernate.impl.SessionImpl.flush(SessionImpl.java:1028)

                            at org.hibernate.impl.SessionImpl.managedFlush(SessionImpl.java:366)

                            at org.hibernate.transaction.JDBCTransaction.commit(JDBCTransaction.java:137)

                            at org.hibernate.ejb.TransactionImpl.commit(TransactionImpl.java:54)

                            at com.informatica.repository.persistence.database.DatabaseRepositoryMetadataUtil.persistTargetMappingAndViewSpecUpgradeLogs(DatabaseRepositoryMetadataUtil.java:663)

                            at com.informatica.repository.persistence.database.DDLGenRepositoryConfigurationManagerImpl.deleteRepositoryInternal(DDLGenRepositoryConfigurationManager.java:1642)

                            at com.informatica.repository.persistence.database.DDLGenRepositoryConfigurationManagerImpl.deleteRepository(DDLGenRepositoryConfigurationManager.java:1574)

                            at com.informatica.repository.persistence.database.DatabaseRepositoryHandler.deleteRepository(DatabaseRepositoryHandler.java:269)

                            at com.informatica.repository.persistence.database.DatabasePersister.deleteRepository(DatabasePersister.java:99)

                            at com.informatica.repository.persistence.service.impl.AbstractPersister.handleAdminOperationRequest(AbstractPersister.java:211)

                            at com.informatica.repository.persistence.database.DatabasePersister.handleAdminOperationRequest(DatabasePersister.java:270)

                            at com.informatica.repository.persistence.service.impl.PersistenceImpl.handleAdminOperation(PersistenceImpl.java:426)

                            at com.informatica.repository.persistence.service.impl.PersistenceImpl.handleAdminOperation(PersistenceImpl.java:433)

                            at com.informatica.repository.persistence.service.impl.PersistenceImpl.deleteRepositoryAdmin(PersistenceImpl.java:509)

                            at com.informatica.repository.persistence.service.impl.PersistenceAdminImpl.deleteRepository(PersistenceAdminImpl.java:295)

                            at com.informatica.repository.service.capability.impl.repository.RepositoryAdminCapabilityImpl.deleteRepository(RepositoryAdminCapabilityImpl.java:203)

                            at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

                            at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

                            at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

                            at java.lang.reflect.Method.invoke(Method.java:606)

                            at com.informatica.repository.service.impl.RepositoryServiceFacadeProxy.invoke(RepositoryServiceFacadeProxy.java:183)

                            at com.sun.proxy.$Proxy9.deleteRepository(Unknown Source)

                            at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

                            at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

                            at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

                            at java.lang.reflect.Method.invoke(Method.java:606)

                            at com.informatica.repository.client.impl.SessionRepositoryServiceProxyImpl.invoke(SessionRepositoryServiceProxyImpl.java:120)

                            at com.sun.proxy.$Proxy11.deleteRepository(Unknown Source)

                            at com.informatica.repository.client.impl.RepositoryAdminSessionImpl.deleteRepository(RepositoryAdminSessionImpl.java:144)

                            at com.informatica.project.repository.service.capability.repository.impl.ProjectRepositoryAdminCapability.deleteRepository(ProjectRepositoryAdminCapability.java:399)

                            at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

                            at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

                            at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

                            at java.lang.reflect.Method.invoke(Method.java:606)

                            at com.informatica.project.repository.service.impl.ProjectRepoServiceFacadeProxy.resilientMethodInvocation(ProjectRepoServiceFacadeProxy.java:246)

                            at com.informatica.project.repository.service.impl.ProjectRepoServiceFacadeProxy.invoke(ProjectRepoServiceFacadeProxy.java:188)

                            at com.sun.proxy.$Proxy8.deleteRepository(Unknown Source)

                            at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

                            at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

                            at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

                            at java.lang.reflect.Method.invoke(Method.java:606)

                            at com.informatica.project.repository.client.impl.SessionSvcFacadeWrapper.resilientMethodInvocation(SessionSvcFacadeWrapper.java:232)

                            at com.informatica.project.repository.client.impl.SessionSvcFacadeWrapper.invoke(SessionSvcFacadeWrapper.java:160)

                            at com.sun.proxy.$Proxy14.deleteRepository(Unknown Source)

                    at com.informatica.project.repository.client.impl.ProjectRepoAdminSessionImpl.deleteRepository(ProjectRepoAdminSessionImpl.java:533)

                            at com.informatica.isp.domainrepositoryservice.impl.PRSDomainRepositoryServiceImpl.deleteDBRepository(PRSDomainRepositoryServiceImpl.java:170)

                            at com.informatica.isp.infasetup.impl.BackupRestorePluginImpl.restoreDomain(BackupRestorePluginImpl.java:269)

                            at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

                            at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

                            at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

                            at java.lang.reflect.Method.invoke(Method.java:606)

                            at com.informatica.pcsf.cmdframework.PluginCommand.run(PluginCommand.java:46)

                            at com.informatica.pcsf.cmdframework.Infasetup.execute(Infasetup.java:224)

                            at com.informatica.pcsf.cmdframework.Infasetup.main(Infasetup.java:87)

                    Caused by: java.sql.SQLException: [informatica][Oracle JDBC Driver][Oracle]ORA-25153: Temporary Tablespace is Empty

                     

                     

                            at com.informatica.jdbc.oraclebase.ddcr.b(Unknown Source)

                            at com.informatica.jdbc.oraclebase.ddcr.a(Unknown Source)

                            at com.informatica.jdbc.oraclebase.ddcq.b(Unknown Source)

                            at com.informatica.jdbc.oraclebase.ddcq.a(Unknown Source)

                            at com.informatica.jdbc.oracle.ddag.<init>(Unknown Source)

                            at com.informatica.jdbc.oracle.ddam.a(Unknown Source)

                            at com.informatica.jdbc.oracle.ddam.a(Unknown Source)

                            at com.informatica.jdbc.oracle.ddam.a(Unknown Source)

                            at com.informatica.jdbc.oracle.ddam.t(Unknown Source)

                            at com.informatica.jdbc.oraclebase.dde3.w(Unknown Source)

                            at com.informatica.jdbc.oraclebase.dde3.y(Unknown Source)

                            at com.informatica.jdbc.oraclebase.dddz.executeUpdate(Unknown Source)

                            at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeUpdate(NewProxyPreparedStatement.java:105)

                            at org.hibernate.jdbc.NonBatchingBatcher.addToBatch(NonBatchingBatcher.java:46)

                            at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:2265)

                    • 7. Re: Unable to bring up Informatica services in 9.6.1
                      user101600 Guru

                      In regards to this error

                      [ICMD_10033] Command [RestoreDomain] failed with error [[INFASETUP_10039] Cannot restore the domain because of the following error: [PRSVCSHARED_00006] Repository ['ISP-Repository'] contains content. Before creating content for a repository, verify that no content exists..]

                      This will occur when youre trying to restore the repo into a DB schema that is not empty and contains a repository already in it.

                      When doing a restore it needs to go into an empty repo schema.

                      Maybe these two KB links will help guide you.

                      https://kb.informatica.com/solution/23/Pages/70/577025.aspx

                      https://kb.informatica.com/howto/6/Pages/17/338325.aspx

                      • 8. Re: Unable to bring up Informatica services in 9.6.1
                        Nico Heinze Guru

                        Hi Sudharsan,

                         

                        Obviously you're working with Oracle as the repository DBMS, but which version is it?

                        Could you please ask your DBA what in other cases usually is the reason for this error message?

                         

                        Regards,

                        Nico