RAC在OCR磁盤故障且有備份時的恢復操作

問題描寫：

CRS啟動失敗，并且RAC進程全部退出。

查看messages日志：
... ...
Apr 10 15:58:31 node74 logger: Cluster Ready Services waiting on dependencies. Diagnostics in /tmp/crsctl.7510.
Apr 10 15:58:31 node74 logger: Cluster Ready Services waiting on dependencies. Diagnostics in /tmp/crsctl.7264.
Apr 10 15:58:31 node74 logger: Cluster Ready Services waiting on dependencies. Diagnostics in /tmp/crsctl.7174.
Apr 10 15:59:31 node74 su: (to oracle) root on none
Apr 10 15:59:31 node74 su: (to oracle) root on none
Apr 10 15:59:31 node74 su: (to oracle) root on none
Apr 10 15:59:31 node74 logger: Cluster Ready Services waiting on dependencies. Diagnostics in /tmp/crsctl.7510.
Apr 10 15:59:31 node74 logger: Cluster Ready Services waiting on dependencies. Diagnostics in /tmp/crsctl.7174.
Apr 10 15:59:31 node74 logger: Cluster Ready Services waiting on dependencies. Diagnostics in /tmp/crsctl.7264.
Apr 10 16:00:26 node74 su: (to oracle) root on /dev/pts/0
Apr 10 16:00:31 node74 su: (to oracle) root on none
Apr 10 16:00:31 node74 su: (to oracle) root on none
Apr 10 16:00:31 node74 su: (to oracle) root on none
Apr 10 16:00:31 node74 logger: Cluster Ready Services waiting on dependencies. Diagnostics in /tmp/crsctl.7264.
Apr 10 16:00:31 node74 logger: Cluster Ready Services waiting on dependencies. Diagnostics in /tmp/crsctl.7510.
Apr 10 16:00:31 node74 logger: Cluster Ready Services waiting on dependencies. Diagnostics in /tmp/crsctl.7174.
... ...

取其中1個文件下來，/tmp/crsctl.7174：

Oracle Cluster Registry initialization failed with invalid format: PROC⑵2: The OCR backend has an invalid format

懷疑是OCR破壞。運行crsctl check boot命令，與上面的日志報錯是1致的：

oracle@node74:~> crsctl check boot
Oracle Cluster Registry initialization failed with invalid format: PROC⑵2: The OCR backend has an invalid format

問題處理：

查看是不是有OCR自動備份，很不幸，沒有結果輸出：

oracle@node74:~> ocrconfig -showbackup
---空

幸虧之前做OCR遷移實驗時，有做過OCR的手動備份。

node74:/home/database/oracle/oracrs/bin # ll /home/oracle/ocr.exp
-rw-r--r-- 1 root root 87246 Dec 29 16:45 /home/oracle/ocr.exp

導入之前備份的OCR：

node74:/home/database/oracle/oracrs/bin # ./ocrconfig -import /home/oracle/ocr.exp

檢查OCR：

node74:/home/database/oracle/oracrs/bin # ./ocrcheck
Status of Oracle Cluster Registry is as follows :
         Version                  :          2
         Total space (kbytes)     :     975584
         Used space (kbytes)      :       3844
         Available space (kbytes) :     971740
         ID                       :  811637621
         Device/File Name         : /dev/raw/raw1
                                    Device/File integrity check succeeded

                                    Device/File not configured

         Cluster registry integrity check succeeded

         Logical corruption check succeeded

---修復成功

啟動CRS：         

node74:/home/database/oracle/oracrs/bin # /etc/init.d/init.crs start

所有RAC進程啟動成功：

node74:~ # su - oracle
oracle@node74:~> crs_stat -t
Name           Type           Target    State     Host        
------------------------------------------------------------
ora....SM1.asm application    ONLINE    ONLINE    node74      
ora....74.lsnr application    ONLINE    ONLINE    node74      
ora.node74.gsd application    ONLINE    ONLINE    node74      
ora.node74.ons application    ONLINE    ONLINE    node74      
ora.node74.vip application    ONLINE    ONLINE    node74      
ora....SM2.asm application    ONLINE    ONLINE    node76      
ora....76.lsnr application    ONLINE    ONLINE    node76      
ora.node76.gsd application    ONLINE    ONLINE    node76      
ora.node76.ons application    ONLINE    ONLINE    node76      
ora.node76.vip application    ONLINE    ONLINE    node76      
ora.orcl.db    application    ONLINE    ONLINE    node74      
ora....l1.inst application    ONLINE    ONLINE    node74      

ora....l2.inst application    ONLINE    ONLINE    node76   

--問題解決