This post is also available in: English
Antes da versão Oracle 11gR2, só era possível termos os arquivos de OCR e Voting Disk em múltiplos raw devices, compartilhados entre os nós de um Cluster.
A partir desta versão, se torna bem interessante aproveitarmos dos recursos de mirror e striping do ASM para colocarmos estes arquivos.
Para colocarmos o OCR e Voting Disk em um diskgroup, devemos obedecer as seguintes regras no momento de definição:
- Se o seu diskgroup tive redundância "external", será necessário que ele tenha no mínimo 1 disco associado.
- Se o seu diskgroup tive redundância "normal", será necessário que ele tenha no mínimo 3 discos associados, mesmo sendo possível criar esse diskgroup com apenas 2 discos.
- Se o seu diskgroup tive redundância "high", será necessário que ele tenha no mínimo 5 discos associados, mesmo sendo possível criar esse diskgroup com apenas 3 discos.
Neste artigo, irei utilizar a segunda opção, de redundância normal. Como no meu servidor só existem 2 discos disponíveis para a criação, precisei solicitar a equipe de Infra a adição de mais um disco de 1 GB:
[root@orarac01 ~]# /etc/init.d/oracleasm listdisks DATA01 OCRVOTE01 OCRVOTE02 [root@orarac01 ~]#
O disco já foi disponibilizado:
[root@orarac01 ~]# multipath -ll DATA01 (36000c29ab0accd959da8f1f78dd2de2f) dm-4 VMware,Virtual disk size=300G features='0' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=1 status=active `- 3:0:2:0 sde 8:64 active ready running OCRVOTE03 (36000c299550e110b041c9ab26acd33ac) dm-5 VMware,Virtual disk size=1.0G features='0' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=1 status=active `- 3:0:3:0 sdf 8:80 active ready running OCRVOTE02 (36000c29b67329567f6a6547d6fd3c769) dm-2 VMware,Virtual disk size=1.0G features='0' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=1 status=active `- 3:0:0:0 sdc 8:32 active ready running OCRVOTE01 (36000c29c1a57ee047d443991ef48894a) dm-3 VMware,Virtual disk size=1.0G features='0' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=1 status=active `- 3:0:1:0 sdd 8:48 active ready running [root@orarac01 ~]#
Adicionando um disco novo para o oracleasm:
[root@orarac01 ~]# /etc/init.d/oracleasm createdisk OCRVOTE03 /dev/dm-5 Marking disk "OCRVOTE03" as an ASM disk: [ OK ] [root@orarac01 ~]# /etc/init.d/oracleasm listdisks DATA01 OCRVOTE01 OCRVOTE02 OCRVOTE03 [root@orarac01 ~]#
Habilitando o disco no outro nó do RAC:
[root@orarac02 ~]# /etc/init.d/oracleasm scandisks Scanning the system for Oracle ASMLib disks: [ OK ] [root@orarac02 ~]# /etc/init.d/oracleasm listdisks DATA01 OCRVOTE01 OCRVOTE02 OCRVOTE03 [root@orarac02 ~]#
O próximo passo é criarmos o novo diskgroup no ASM.
Primeiro, vamos tentar colocá-los como "High Redundancy" e ver o que acontece.
CREATE DISKGROUP OCRVOTE HIGH REDUNDANCY FAILGROUP OCRVOTE_FG01 DISK '/dev/oracleasm/disks/OCRVOTE01' NAME OCRVOTE01 SIZE 1024M FAILGROUP OCRVOTE_FG02 DISK '/dev/oracleasm/disks/OCRVOTE02' NAME OCRVOTE02 SIZE 1024M FAILGROUP OCRVOTE_FG03 DISK '/dev/oracleasm/disks/OCRVOTE03' NAME OCRVOTE03 SIZE 1024M ATTRIBUTE 'compatible.asm'='11.2.0.0.0','au_size'='1M';
[root@orarac01 ~]# crsctl replace votedisk +OCRVOTE Failed to create voting files on disk group OCRVOTE. Change to configuration failed, but was successfully rolled back. CRS-4000: Command Replace failed, or completed with errors. [root@orarac01 ~]#
[root@orarac01 ~]# tail -n 200 $GRID_HOME/log/orarac01/client/crsctl_root.log ... 2013-06-18 11:46:13.220: [ CSSCLNT][278546160]clsssConfigLockTrace: === clsssConfigLock === 2013-06-18 11:46:13.220: [ CSSCLNT][278546160]clsssConfigLockTrace: lock(0x2838a50), version(1), size(1088) 2013-06-18 11:46:13.220: [ CSSCLNT][278546160]clsssConfigLockTrace: offsets(24), activever(186647296) 2013-06-18 11:46:13.220: [ CSSCLNT][278546160]clsssConfigLockTrace: id(15), instantiation(5), incarn(1) 2013-06-18 11:46:13.220: [ CSSCLNT][278546160]clsssConfigLockTrace: mapoff(28), configoff(548), mapsize(512) 2013-06-18 11:46:13.220: [ CSSCLNT][278546160]clsssConfigLockTrace: state(10), holders(1), waiters(0) 2013-06-18 11:46:13.220: [ CSSCLNT][278546160]clsssConfigLockTrace: committimestamp(0), commitstate(0) 2013-06-18 11:46:13.220: [ CSSCLNT][278546160]clsssConfigLockTrace: === Map (of first 7 entries) === 2013-06-18 11:46:13.220: [ CSSCLNT][278546160]clsssConfigLockTrace: 010 000 000 000 000 000 000 2013-06-18 11:46:13.221: [ CRSCTL][278546160]crsctl_replace_votedisk: calling kgfdvfPushDiskString 2013-06-18 11:46:13.397: [ CRSCTL][278546160]function crsctl_replace_votedisk, error 1 [clsuSlosFormatDiag called with non-error slos.] . 2013-06-18 11:46:13.397: [ CRSCTL][278546160]crsctl_replace_votedisk: pushed the ASM DS to the profile 1 [ CLWAL][278546160]clsw_Initialize: OLR initlevel [30000] 2013-06-18 11:46:13.432: [ CRSCTL][278546160]crsctl_format_diskgroup: creating diskgroup OCRVOTE 2013-06-18 11:46:16.555: [ CRSCTL][278546160]function crsctl_format_diskgroup, error 1 SLOS: cat=-1, opn=(none), dep=(none), loc=ORA-01405: f . 2013-06-18 11:46:16.555: [ CRSCTL][278546160]crsctl_format_diskgroup: diskgroup OCRVOTE creation with status 1. Please check the alert log file for ASM 2013-06-18 11:46:16.568: [ CSSCLNT][278546160]clsssConfigLockTrace: === clsssConfigUnlock === 2013-06-18 11:46:16.568: [ CSSCLNT][278546160]clsssConfigLockTrace: lock(0x2837b50), version(1), size(1088) 2013-06-18 11:46:16.568: [ CSSCLNT][278546160]clsssConfigLockTrace: offsets(24), activever(186647296) 2013-06-18 11:46:16.568: [ CSSCLNT][278546160]clsssConfigLockTrace: id(15), instantiation(5), incarn(3) 2013-06-18 11:46:16.568: [ CSSCLNT][278546160]clsssConfigLockTrace: mapoff(28), configoff(548), mapsize(512) 2013-06-18 11:46:16.568: [ CSSCLNT][278546160]clsssConfigLockTrace: state(0), holders(0), waiters(0) 2013-06-18 11:46:16.568: [ CSSCLNT][278546160]clsssConfigLockTrace: committimestamp(0), commitstate(0) 2013-06-18 11:46:16.568: [ CSSCLNT][278546160]clsssConfigLockTrace: === Map (of first 7 entries) === 2013-06-18 11:46:16.568: [ CSSCLNT][278546160]clsssConfigLockTrace: 000 000 000 000 000 000 000 ...
[root@orarac01 ~]# tail -n 500 /u01/app/grid/diag/asm/+asm/+ASM1/trace/alert_+ASM1.log ... Tue Jun 18 11:46:13 2013 NOTE: Creating voting files in diskgroup OCRVOTE Tue Jun 18 11:46:13 2013 NOTE: Voting File refresh pending for group 2/0x7f386577 (OCRVOTE) NOTE: Attempting voting file creation in diskgroup OCRVOTE NOTE: voting file allocation on grp 2 disk OCRVOTE_0000 NOTE: voting file allocation on grp 2 disk OCRVOTE_0001 NOTE: voting file allocation on grp 2 disk OCRVOTE_0002 ERROR: Voting file allocation failed for group OCRVOTE Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_6301.trc: ORA-15274: Not enough failgroups (5) to create voting files NOTE: Attempting voting file refresh on diskgroup OCRVOTE NOTE: Voting file relocation is required in diskgroup OCRVOTE NOTE: Attempting voting file relocation on diskgroup OCRVOTE NOTE: voting file deletion on grp 2 disk OCRVOTE_0000 NOTE: voting file deletion on grp 2 disk OCRVOTE_0001 NOTE: voting file deletion on grp 2 disk OCRVOTE_0002 ...
O erro é que são necessários pelo menos 5 failgroups para criar o Voting Disk.
Vamos recriá-los então com a redundância correta:
CREATE DISKGROUP OCRVOTE NORMAL REDUNDANCY FAILGROUP OCRVOTE_FG01 DISK '/dev/oracleasm/disks/OCRVOTE01' NAME OCRVOTE01 SIZE 1024M FAILGROUP OCRVOTE_FG02 DISK '/dev/oracleasm/disks/OCRVOTE02' NAME OCRVOTE02 SIZE 1024M FAILGROUP OCRVOTE_FG03 DISK '/dev/oracleasm/disks/OCRVOTE03' NAME OCRVOTE03 SIZE 1024M ATTRIBUTE 'compatible.asm'='11.2.0.0.0','au_size'='1M';
[root@orarac01 ~]# crsctl replace votedisk +OCRVOTE Successful addition of voting disk 05ca1320e3244f70bf45d1fc37d325d5. Successful addition of voting disk 1affc0b6da314fe8bfd8dec18af4923b. Successful addition of voting disk 3d87bf2793e04fe4bfd6764f24ffafb1. Successful deletion of voting disk 4c9ddcb1acd14f27bf77652370a1a5b1. Successfully replaced voting disk group with +OCRVOTE. CRS-4266: Voting file(s) successfully replaced [root@orarac01 ~]# crsctl query css votedisk ## STATE File Universal Id File Name Disk group -- ----- ----------------- --------- --------- 1. ONLINE 05ca1320e3244f70bf45d1fc37d325d5 (/dev/oracleasm/disks/OCRVOTE01) [OCRVOTE] 2. ONLINE 1affc0b6da314fe8bfd8dec18af4923b (/dev/oracleasm/disks/OCRVOTE02) [OCRVOTE] 3. ONLINE 3d87bf2793e04fe4bfd6764f24ffafb1 (/dev/oracleasm/disks/OCRVOTE03) [OCRVOTE] Located 3 voting disk(s). [root@orarac01 ~]#
O Voting Disk já foi.
O próximo passo é o CRS. Note que estamos transferindo ele de outro diskgroup (+DATA), que é utilizado para guardar datafiles, para este recém criado:
[root@orarac01 ~]# ocrconfig -replace +DATA -replacement +OCRVOTE PROT-28: Cannot delete or replace the only configured Oracle Cluster Registry location [root@orarac01 ~]#
Por existir apenas um destino OCR configurado, o comando "replace" não funciona. Vamos precisar adicionar um outro e apagar o antigo:
[root@orarac01 ~]# ocrconfig -add +OCRVOTE [root@orarac01 ~]# ocrcheck Status of Oracle Cluster Registry is as follows : Version : 3 Total space (kbytes) : 262120 Used space (kbytes) : 2836 Available space (kbytes) : 259284 ID : 1089167638 Device/File Name : +DATA Device/File integrity check succeeded Device/File Name : +OCRVOTE Device/File integrity check succeeded Device/File not configured Device/File not configured Device/File not configured Cluster registry integrity check succeeded Logical corruption check succeeded [root@orarac01 ~]#
Agora apagamos o antigo:
[root@orarac01 ~]# ocrconfig -delete +DATA [root@orarac01 ~]# ocrcheck Status of Oracle Cluster Registry is as follows : Version : 3 Total space (kbytes) : 262120 Used space (kbytes) : 2836 Available space (kbytes) : 259284 ID : 1089167638 Device/File Name : +OCRVOTE Device/File integrity check succeeded Device/File not configured Device/File not configured Device/File not configured Device/File not configured Cluster registry integrity check succeeded Logical corruption check succeeded [root@orarac01 ~]# cat /etc/oracle/ocr.loc #Device/file +DATA getting replaced by device +OCRVOTE ocrconfig_loc=+OCRVOTE local_only=false [root@orarac01 ~]#
Pronto!
Mais informações podem ser obtidas no Artigo do Metalink abaixo:
OCR / Vote disk Maintenance Operations: (ADD/REMOVE/REPLACE/MOVE) [ID 428681.1]