UDEV ASM Linux
1.1 Oracle block devices
Due to the fact that RH will not be supporting raw devices after enterprise linux 5 Oracle now recommend using block devices instead of raw devices.
The following instructions for setting up the ASM devices is taken from the Oracle technical note number 602952.1
Â
1.1.1 Partition devices
Identify the sd devices to be used by asm.
“cat /proc/partitions†will list all devices, look at the “sd“ devices.
e.g.
# cat /proc/partitions
major minor #blocks name
8Â Â Â Â 0Â Â Â 20971520 sda
8Â Â Â Â 0Â Â Â Â Â Â Â 104391 sda1
8Â Â Â Â 0Â Â Â 20860402 sda2
8Â Â Â 16Â Â 12582912 sdb
8Â Â Â 32Â Â 12582912 sdc
8Â Â Â 48Â Â Â Â 1048576 sdd
8Â Â Â 64Â Â Â Â 1048576 sde
8Â Â Â 64Â Â Â Â 1048576 sdf
8Â Â Â 64Â Â Â Â 1048576 sdg
8Â Â Â 64Â Â Â Â 1048576 sdh
Here the ASM volumes are sdb and sdc (each 12Gb)
The OCR and Voting disks are sdd to sdh (each 1Gb)
Now we have identified the disks we need to create partitions on them.
For each device enter:
$ fdisk /dev/sd-??
Command (m for help): u
Command (m for help): p
Command (m for help): n
Command (m for help): p
1
32768
w
(Where ?? are the characters used to identify the device.)
The reason we are using 32768 as the start block for the device is because of the way scsi/san volume mapping uses the first few blocks of a device and subsequently causes excessive write I/O. This is better explained in the following link:
http://www.pythian.com/news/411/aligning-asm-disks-on-linux
e.g.
# fdisk /dev/sdb
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won’t be recoverable.
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
Command (m for help): u
Changing display/entry units to sectors
Command (m for help): p
Disk /dev/dm-5: 5394 MB, 5394923520 bytes
255 heads, 63 sectors/track, 655 cylinders, total 10536960 sectors
Units = sectors of 1 * 512 = 512 bytes
Device Boot     Start        End     Blocks  Id System
Command (m for help): n
Command action
e  extended
p  primary partition (1-4)
p
Partition number (1-4): 1
First sector (63-10536959, default 63): 32768
Last sector or +size or +sizeM or +sizeK (32768-25165823, default 25165823):
Using default value 25165823
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
#
1.2 Configure device protection
Create or edit the file /etc/udev/rules.d/98-oracle-perm.rules and add the following:
#OCR DISKS
KERNEL==â€sdb*â€, GROUP=â€oinstallâ€, MODE=â€640â€
KERNEL==â€sdc*â€, GROUP=â€oinstallâ€, MODE=â€640â€
#VOTE DISKS
KERNEL==â€sdd*â€, OWNER=â€oracleâ€, GROUP=â€oinstallâ€, MODE=â€660â€
KERNEL==â€sde*â€, OWNER=â€oracleâ€, GROUP=â€oinstallâ€, MODE=â€660â€
#ASM DISKS
KERNEL==â€sdf*â€, OWNER=â€oracleâ€, GROUP=â€oinstallâ€, MODE=â€660â€
KERNEL==â€sdg*â€, OWNER=â€oracleâ€, GROUP=â€oinstallâ€, MODE=â€660â€
KERNEL==â€sdh*â€, OWNER=â€oracleâ€, GROUP=â€oinstallâ€, MODE=â€660â€
Changing the sd device name where necessary.
Â
Create or edit the file /etc/udev/rules.d/99-oracle-naming.rules and add the following:
KERNEL==â€sdb1â€, NAME=â€ocr1â€
KERNEL==â€sdc1â€, NAME=â€ocr2â€
KERNEL==â€sdd1â€, NAME=â€vote1â€
KERNEL==â€sde1â€, NAME=â€vote2â€
KERNEL==â€sdf1â€, NAME=â€vote3â€
KERNEL==â€sdg1â€, NAME=â€asm2â€
KERNEL==â€sdh1â€, NAME=â€asm1â€
Changing the sd device name where necessary.
Â
Reboot the system.
After the reboot we should see the asm, ocr and vote volumes in /dev.
Â
1.2.1 Map the asm volumes
For each of the asm volumes execute the command:
/etc/init.d/oracleasm createdisk ASMn /dev/asmn
Replacing the “n†with the relevant numbers.
Perform the following on all nodes in the cluster:
# /etc/init.d/oracleasm scandisks
# /etc/init.d/oracleasm listdisks
The ASM volumes should now be listed in the directory /dev/oracleasm/disks
# ls /dev/oracleasm/disks
ASM1Â Â Â Â ASM2Â Â Â ASM3 etc
Â
ALTERNATIVE COMMANDS
================
You have to add the disk information to the below rules file on all nodes
# view /etc/udev/rules.d/99-oracle-asmdevices.rules
Ex entry
KERNEL==”dm-[0-9]*”, ENV{DM_UUID}==”mpath-360060e801526af00000126af00001c2b”, NAME=”ISP_A_ARCH_10″, OWNER=”oracle”, GROUP=”asmdba”, MODE=”0660″
Below information for the above line will be available in /etc/multipath.conf file
360060e801526af00000126af00001c2b
ISP_A_ARCH_10
Reload and Restart the udev rules on all nodes
# udevadm control –reload-rules
# start_udev
Â
When you look the /dev/mapper directory; you can see the new disk added and you can see the disk the asmca
Â
Discussion ¬