Creating the 11.2 OEM (with Oracle Flex) with emca Fails with Message: WARNING: Error determining the nodes associated with cluster database for ASM

Hi,

When I try to create Oracle Enterprise Manager with EMCA for 11.2.0.3 database running on 18c Grid Infrastructure, it fails with following error mesaage

Database Control (DBC) 11.2 web site creation with emca fails for Oracle Flex with this error message:

WARNING: Error determining the nodes associated with cluster database for ASM
Apr 9, 2019 11:28:18 PM oracle.sysman.emcp.ParamsManager queryDBForASMNodeList
CONFIG: Stack trace:
PRCA-1018 : Failed to retrieve ASM instance list for ASM asm
PRCR-1115 : Failed to find entities of type resource instance that match filters ((TYPE == ora.asm.type) && (NAME == ora.asm)) and contain attributes LAST_SERVER,USR_ORA_INST_NAME
CRS-2653: Special value evaluation must be associated with a resource instance
at oracle.cluster.impl.asm.ASMImpl.getInstances(ASMImpl.java:787)
at oracle.cluster.impl.asm.ASMImpl.allInstances(ASMImpl.java:702)
at oracle.sysman.emcp.ParamsManager.queryDBForASMNodeList(ParamsManager.java:1899)
at oracle.sysman.emcp.ParamsManager.getASMNodeList(ParamsManager.java:1673)
at oracle.sysman.emcp.ParamsManager.validatePassword(ParamsManager.java:3483)
at oracle.sysman.emcp.EMConfigAssistant.promptForData(EMConfigAssistant.java:710)
at oracle.sysman.emcp.EMConfigAssistant.promptForParams(EMConfigAssistant.java:3028)
at oracle.sysman.emcp.EMConfigAssistant.displayWarnsAndPromptParams(EMConfigAssistant.java:3065)
at oracle.sysman.emcp.EMConfigAssistant.getDisplayAndPromptWarnsParms(EMConfigAssistant.java:3090)
at oracle.sysman.emcp.EMConfigAssistant.performConfiguration(EMConfigAssistant.java:1116)
at oracle.sysman.emcp.EMConfigAssistant.statusMain(EMConfigAssistant.java:583)
at oracle.sysman.emcp.EMConfigAssistant.main(EMConfigAssistant.java:529)
Caused by: PRCR-1115 : Failed to find entities of type resource instance that match filters ((TYPE == ora.asm.type) && (NAME == ora.asm)) and contain attributes LAST_SERVER,USR_ORA_INST_NAME
CRS-2653: Special value evaluation must be associated with a resource instance
at oracle.cluster.impl.crs.CRSNative.searchEntities(CRSNative.java:1016)
at oracle.cluster.impl.crs.CRSFactoryImpl.searchEntities(CRSFactoryImpl.java:637)
at oracle.cluster.impl.asm.ASMImpl.getInstances(ASMImpl.java:768)
... 11 more
Caused by: CRS-2653: Special value evaluation must be associated with a resource instance
at oracle.cluster.impl.crs.CRSNativeResult.addComp(CRSNativeResult.java:171)
at oracle.cluster.impl.crs.CRSNative.doQueryEntities(Native Method)
at oracle.cluster.impl.crs.CRSNative.searchEntities(CRSNative.java:1006)
... 13 more
Apr 9, 2019 11:28:18 PM oracle.sysman.emcp.ParamsManager getInaccessibleSidList
CONFIG: Invalid Node/Sid List specified.

This is due to bug number 20416112.

In the Oracle Flex ASM configuration not all ASM instances are required to be Up or even to have been yet started. In such a case, the DBC web site configuration process can fail.

Querying the ASM instances for activity shows the following, namely one OFFLINE ASM instance.

orcnode1 /home/oracle> crsctl stat res ora.asm -t 
--------------------------------------------------------------------------
Name Target State Server State details 
--------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------
ora.asm
1 ONLINE ONLINE orcnode1 Started,STABLE
2 ONLINE ONLINE orcnode2 Started,STABLE
3 OFFLINE OFFLINE STABLE
--------------------------------------------------------------------------

Please also check clsuter mode.

orcnode1 /home/oracle> asmcmd showclustermode
ASM cluster : Flex mode enabled – Direct Storage Access
orcnode1 /home/oracle> srvctl config asm
ASM home: <CRS home>
Password file: +DATA/orapwASM
Backup of Password file:
ASM listener: LISTENER_REP
ASM instance count: 3
Cluster ASM listener: ASMNET1LSNR_ASM,ASMNET2LSNR_ASM

For workaround, please issue this command with grid os user.
orcnode1 /home/oracle> srvctl modify asm -count ALL

And again check status.

orcnode1 /home/oracle> srvctl config asm
ASM home: <CRS home>
Password file: +DATA/orapwASM
Backup of Password file:
ASM listener: LISTENER_REP
ASM instance count: ALL
Cluster ASM listener: ASMNET1LSNR_ASM,ASMNET2LSNR_ASM

orcnode1 /home/oracle> crsctl stat res ora.asm -t
--------------------------------------------------------------------------
Name Target State Server State details 
--------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------
ora.asm
1 ONLINE ONLINE orcnode1 Started,STABLE
2 ONLINE ONLINE orcnode2 Started,STABLE
--------------------------------------------------------------------------
orcnode1 /home/oracle>

This action will decrease the number of ASM instances expected by emca to the number of ONLINE instances returned by  the “crsctl stat res ora.asm -t” command and allow the configuration to continue.

Now, you can retry to run EMCA with cluster parameter.

How To Manually Install Oracle OEM 13c Agent?

1- We need to download the agent software from OMS Server for the platform which we will install. Login with sysman.

[oracle@isu13c bin]$ ./emcli login -username=sysman
Enter password :

Login successful

[oracle@isu13c bin]$ ./emcli sync
Synchronized successfully

2- Get the supported platform list

[oracle@isu13c bin]$ ./emcli get_supported_platforms
———————————————–
Version = 13.1.0.0.0
Platform = Linux x86-64
———————————————–
Platforms list displayed successfully.

3- Download the agent software

[oracle@isu13c bin]$ ./emcli get_agentimage -destination=/tmp/agentinstaller -platform=”Linux x86-64″
=== Partition Detail ===
Space free : 53 GB
Space required : 1 GB
Check the logs at /u01/app/oracle/gc_inst/em/EMGC_OMS1/sysman/emcli/setup/.emcli/get_agentimage_2018-04-25_12-12-30-PM.log
Downloading /tmp/agentinstaller/13.1.0.0.0_AgentCore_226.zip
File saved as /tmp/agentinstaller/13.1.0.0.0_AgentCore_226.zip
Downloading /tmp/agentinstaller/13.1.0.0.0_Plugins_226.zip
File saved as /tmp/agentinstaller/13.1.0.0.0_Plugins_226.zip
Downloading /tmp/agentinstaller/unzip
File saved as /tmp/agentinstaller/unzip
Executing command: /tmp/agentinstaller/unzip /tmp/agentinstaller/13.1.0.0.0_Plugins_226.zip -d /tmp/agentinstaller
Exit status is:0
Agent Image Download completed successfully.

4- let me copy the software to target server and unzip.

scp 13.1.0.0.0_AgentCore_226.zip oracle@172.16.3.29:/DATA/install

unzip 13.1.0.0.0_AgentCore_226.zip

5- Let me install the agent

mkdir /u01/app/oracle/agent
./agentDeploy.sh AGENT_BASE_DIR=/u01/app/oracle/agent \
-force \
-ignorePrereqs \
-invPtrLoc /etc/oraInst.loc \
AGENT_PORT=3872 \
EM_UPLOAD_PORT=4903 \
OMS_HOST=isu13c \
ORACLE_HOSTNAME=oracledrs \
AGENT_INSTANCE_HOME=/u01/app/oracle/agent/agent_inst \
AGENT_REGISTRATION_PASSWORD=xxxxxx \
SCRATCHPATH=/tmp

After installation, don’t forget to execute root.sh script as root user.

Our agent is running and ready.

/u01/app/oracle/agent/agent_13.1.0.0.0/bin/emctl status agent
Oracle Enterprise Manager Cloud Control 13c Release 1
Copyright (c) 1996, 2015 Oracle Corporation. All rights reserved.
—————————————————————
Agent Version : 13.1.0.0.0
OMS Version : 13.1.0.0.0
Protocol Version : 12.1.0.1.0
Agent Home : /u01/app/oracle/agent/agent_inst
Agent Log Directory : /u01/app/oracle/agent/agent_inst/sysman/log
Agent Binaries : /u01/app/oracle/agent/agent_13.1.0.0.0
Core JAR Location : /u01/app/oracle/agent/agent_13.1.0.0.0/jlib
Agent Process ID : 64876
Parent Process ID : 64751
Agent URL : https://oracledrs:3872/emd/main/
Local Agent URL in NAT : https://oracledrs:3872/emd/main/
Repository URL : https://isu13c.localdomain:4903/empbs/upload
Started at : 2017-11-21 09:51:57
Started by user : oracle
Operating System : Linux version 4.1.12-37.4.1.el6uek.x86_64 (amd64)
Number of Targets : 7
Last Reload : (none)
Last successful upload : 2018-04-25 13:22:43
Last attempted upload : 2018-04-25 13:22:43
Total Megabytes of XML files uploaded so far : 172.1
Number of XML files pending upload : 0
Size of XML files pending upload(MB) : 0
Available disk space on upload filesystem : 59.73%
Collection Status : Collections enabled
Heartbeat Status : Ok
Last attempted heartbeat to OMS : 2018-04-25 13:23:53
Last successful heartbeat to OMS : 2018-04-25 13:23:53
Next scheduled heartbeat to OMS : 2018-04-25 13:24:53

—————————————————————
Agent is Running and Ready

How to Change Lost SYSMAN Password On OEM Cloud Control 13c?

Hi,

To change lost SYSMAN password on Oracle Enterprise Manager 13c, follow the below steps:

1- Change the password from SQL*Plus

alter user sysman identified by “oracle_test”;

2-Go to middleware home directory

cd /u01/app/oracle/middleware/bin/
./emctl config oms -list_repos_details
Oracle Enterprise Manager Cloud Control 13c Release 1
Copyright (c) 1996, 2015 Oracle Corporation. All rights reserved.
Repository Connect Descriptor : (DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=oem13c)(PORT=1521)))(CONNECT_DATA=(SID=emrep)))
Repository User : SYSMAN

3- Change the password as the following command

./emctl config oms -change_repos_pwd -use_sys_pwd -sys_pwd oracle_test -new_pwd oracle_test
Oracle Enterprise Manager Cloud Control 13c Release 1
Copyright (c) 1996, 2015 Oracle Corporation. All rights reserved.

Changing passwords in backend …
Passwords changed in backend successfully.
Updating repository password in Credential Store…
Successfully updated Repository password in Credential Store.
Restart all the OMSs using ’emctl stop oms -all’ and ’emctl start oms’.
Successfully changed repository password.

4- Restart the OMS

./emctl stop oms -all
Oracle Enterprise Manager Cloud Control 13c Release 1
Copyright (c) 1996, 2015 Oracle Corporation. All rights reserved.
Stopping Oracle Management Server…
WebTier Successfully Stopped
Oracle Management Server Successfully Stopped
Oracle Management Server is Down
JVMD Engine is Down
Stopping BI Publisher Server…
BI Publisher Server Successfully Stopped
AdminServer Successfully Stopped
BI Publisher Server is Down

./emctl start oms
Oracle Enterprise Manager Cloud Control 13c Release 1
Copyright (c) 1996, 2015 Oracle Corporation. All rights reserved.
Starting Oracle Management Server…
WebTier Successfully Started
Oracle Management Server Successfully Started
Oracle Management Server is Up
JVMD Engine is Up
Starting BI Publisher Server …
BI Publisher Server Successfully Started
BI Publisher Server is Up

How to Resolve ‘Null Pointer Exception’ Error On Top Activity Page In Oracle Enterprise Manager 13c

Hi,

When you login  Oracle Enterprise Manager 13c, the top activity page shows ‘Null Pointer Exception’ errors.   This is due to bug number 25455462. You can do following steps to resolve this issue.

oem13c_error

1-Set the following property on OEM 13c machine

emctl set property -name use_pooled_target_connections -value false

2-Restart the oms

cd <OMS_HOME>/bin

(cd /u01/app/oracle/middleware/bin/)

./emctl stop oms -all
Oracle Enterprise Manager Cloud Control 13c Release 1
Copyright (c) 1996, 2015 Oracle Corporation. All rights reserved.
Stopping Oracle Management Server…
WebTier Successfully Stopped
Oracle Management Server Successfully Stopped
Oracle Management Server is Down
JVMD Engine is Down
Stopping BI Publisher Server…
BI Publisher Server Successfully Stopped
AdminServer Successfully Stopped
BI Publisher Server is Down

./emctl start oms
Oracle Enterprise Manager Cloud Control 13c Release 1
Copyright (c) 1996, 2015 Oracle Corporation. All rights reserved.
Starting Oracle Management Server…
WebTier Successfully Started
Oracle Management Server Successfully Started
Oracle Management Server is Up
JVMD Engine is Up
Starting BI Publisher Server …
BI Publisher Server Successfully Started
BI Publisher Server is Up

Creating OEM Repository and ORA-00955/ORA-01921 Errors

Hi Friends,

A month ago, I wrote an article about OEM configuration when the database hostname or ip address changed. You can find that in this link https://taliphakanozturken.wordpress.com/2011/12/31/oem-configuration-when-the-database-host-name-or-ip-address-changes/ .

We can get an error while re-creating (emca -config dbcontrol db -repos create) OEM repository as below:

SEVERE: Error creating the repository

Refer to the log file at /oracle/cfgtoollogs/emca/TALIPDB/emca_2012_01_20_12_07_21.log for more details.

Could not complete the configuration. Refer to the log file at /oracle/cfgtoollogs/emca/TALIPDB/emca_2012_01_20_12_07_21.log for more details.

The error is also seen in the log file of errors is analyzed you can see following lines (/oracle/cfgtoollogs/emca/TALIPDB/emca_2012_01_20_12_07_21.log).

CONFIG: ORA-00955: name is already used by an existing object
CONFIG: ORA-01921: role name ‘MGMT_USER’ conflicts with another user or role name

In this case we must do following operations in SQL*Plus.

SQL> DROP USER SYSMAN CASCADE;

SQL> DROP ROLE MGMT_USER;

SQL> DROP USER MGMT_VIEW;

SQL> spool synonym.sql

SQL> select ‘drop public synonym ‘||object_name||’;’ from dba_objects where object_type=’SYNONYM’ and and object_name like ‘MGMT_%’ and status=’INVALID’;

SQL> spool off

SQL> @synonym.sql

Now, please retry to create OEM respository. It will be success.

emca -config dbcontrol db -repos create

 

Talip Hakan Öztürk