Using 11gR2 Grid Infrastructure and ASM with a 10gR2 database

If you installed 11gR2 Grid Infrastructure and ASM, but still have a 10gR2 database. You will want to know a few things to get your environment up and running. I've listed below the gotchas that stalled me for several hours.

• When you install Grid Infrastructure (GI) on a single node, it's called Oracle Restart.
  • Some features of the CRS software don't work. Specifically, you cannot add an instance to the Oracle Cluster Registry (OCR). Only databases.
• You will have to manually add your database to the OCR
  • srvctl add database -d <db unique name> -o <path to oracle home>
• For the database to startup after ASM, you need to add dependencies on the ASM disk groups that it uses
  • srvctl modify database -d <db unique name> -a 'DATA1,DATA2'
  • /etc/oratab is no longer used. CRS will set the startup flag to N in this file every time it starts your database.
• Health Checks
  • If CRS starts your db, but it cannot read the health check file, it will shutdown abort your instance.
  • The health check file is located in $ORACLE_HOME/dbs
    • named hc_<sid>.dat
  • GIMH: GIM-00104: Health check failed to connect to instance.
    • If you get this error, CRS will shutdown abort the database. Look in the log for more error messages to determine why the health check failed.
    • There are issues with having this file on an OCFS volume (which do not support memory mapped files), so the workaround is to move the file to another disk and put a symbolic link in its place.
  • GIM-00090: OS-dependent operation: SMR file size check failed with status: 0
    • I believe this error means the hc_<sid>.data file is corrupted. Rename it and try starting the database again. Worked for me!
    • From metalink:

      • SMR stands for shared memory region and in 10g we have this instance monitoring segment which is used to store data for health monitoring. Every process which starts up attaches to this and the data in this segment is visible to to whoever has this SMR as part of its address space. This gets called during SGA initialization for creation of the shared memory segment and attaching to it.
        
        - The SMR name is constructed from the Oracle Home Name and the SID
        - The file which this data is stored in is $ORACLE_HOME/dbs/hc_SID.dat
        - If the file does not exist we create it.
        - If the file exists then we open and lock it.
        - We check if we have permissions to write to this disk. 
        - Write and seek on the file to verify we can change its size.
        - Then set the appropriate protection mode and then use mmap64 call to map this file to the process address space.
        - If we were creating the SMR then we fill it up with zeroes then lock it
        - If we return a failure on any side then we might have had problems with attaching to the shared memory segment or with the     permissions on the hc_SID.dat file.
        Solution
        
        Things that can be attempted to eliminate these errors :
        
        a) The ORACLE_HOME and ORACLE_SID values are correctly set in the environment
        b) Proper permissions on ths $ORACLE_HOME/dbs and the Oracle user's ability to create files in this directory.
        c) Check for hc_<SID>.dat files within the $ORACLE_HOME/dbs directory
        d) Permissions on the hc_<SID>.dat file and the shmmax , shmmni settings on this Node so that we can create additional  Memory segments for SMR.
        
        You can also try renaming that $ORACLE_HOME/dbs/hc_<$ORACLE_SID>.dat file and then attempt
        to restart the instance to check if this still reports that same SMR warnings.
        
        $ mv $ORACLE_HOME/dbs/hc_<$ORACLE_SID>.dat $ORACLE_HOME/dbs/hc_<$ORACLE_SID>.dat.OLD
        That backup file .OLD can later be removed if needed. 

• Log file locations
  • CRS agent log:
    • $ORACLE_HOME/log/<nodename>/agent/ohasd/oraagent_oracle/oraagent_oracle.log
  • Root agent
    • $ORACLE_HOME/log/<hostname>/agent/ohasd/orarootagent_oracle/orarootagent_oracle.log
  • CSS Daemon agent
    • $ORACLE_HOME/log/<hostname>/agent/ohasd/oracssdagent_oracle/oracssdagent_oracle.log
  • Other miscellaneous logs:
    • $ORACLE_HOME/log/<hostname>/diskmon/client.log
    • $ORACLE_HOME/log/<hostname>/diskmon/diskmon.log
    • $ORACLE_HOME/log/<hostname>/cssd/ocssd.log
    • $ORACLE_HOME/log/<hostname>/cssd/cssdOUT.log
    • $ORACLE_HOME/log/<hostname>/client/crsctl.log
• Manual startup and shutdown
  • You should avoid doing this. Instead, use srvctl start database and srvctl stop database