2012-05-17

Host preferred credentials failure after cloning Agent OEM 11g

Environment
AIX 6.1
Oracle Enterprise Manager (OEM) Grid Control 11.1

Problem
There is an issue with host preferred credentials in Grid Control .
Agent cannot connect to host .
Agent home and database home are owned by different UNIX users.

Cause
OUI is run as part of agent cloning procedure, e.g.:
cd $ORACLE_HOME/oui/bin
./runInstaller -clone -forceClone ORACLE_HOME=/oracle/oem/agent11g ORACLE_HOME_NAME=OH_agent11g -noconfig -silent OMS_HOST=oms.domain.com OMS_PORT=4889

During the cloning OUI creates about a hundred backup files (*.ouibak).
You can see and delete them as:
find $ORACLE_HOME -name "*.ouibak" -ls
find $ORACLE_HOME -name "*.ouibak" -exec rm {} \;

There is a catch here. OUI creates new files with different permissions.
For example in $ORACLE_HOME/perl/bin, all files have 755 before cloning, and 700 after cloning.
If oracle agent and oracle database homes are owned by different UNIX users, even the same group (oinstall), Oracle agent cannot connect to host when you test host preferred credentials in OEM.

When switching the agent into DEBUG mode, you can see in the trace file $ORACLE_HOME/sysman/log/emagent.trc, that the user (database owner) cannot execute perl.
In emagent.nohup, you can see “ERROR: Execution failed due to binary missing or permission issues”.

Solution
At least grant 750 (if owners in the same group - oinstall) or 755 to $OH/perl/bin/perl
Or for all files in $OH/perl
find $ORACLE_HOME/perl -perm 700 -exec chmod g+rx {} \;

Useful info
How to debug agent:
Troubleshooting Host Credentials Issues in Grid Control Console [ID 565414.1]
About perl permission issue:
Testing Host Credentials fails with ERROR: Execution failed due to binary missing or permission issues [ID 1294136.1]

Configuring standby OMS (OEM 11g)


Environment
Oracle Enterprise Manager 11.1
UNIX AIX 6.1

This is a simple configuration. There are two OMS hosts in different co-locations (host1, host2) and common DNS alias (oms).
There is no hardware SLB in this example.

Tips
When you configure OMS with SLB, DNS alias (oms.domain.com) should have different IP address from local hostname.

If they are the same, the result is:
Console Server Host : oms.domain.com
OMS is not configured with SLB or virtual hostname

If they are different, the result is:
Console Server Host : host1.domain.com
SLB or virtual hostname: oms.domain.com

After securing OMS with SLB alias, restart OMS (I didn’t see it in Oracle documentation), otherwise agents will not be able to register with the alias:
emctl secure oms -host oms.domain.com
emctl stop oms -all
emctl start oms

After configuring OMS as SLB/alias, you can point the alias oms.domain.com to host1.

Here is Oracle documentation how to Setup Standby Management Server.