Veritas Volume manager could not see some of the SAN disks with Veritas Cluster

0
360

Veritas Cluster rebooted due to some reason and then could not see some of the SAN drives in the Volume Manager with Solaris OS.
Below Steps were followed for troubleshooting:
1. Check Veritas licences if they were properly present, and they were.
2. Tried to do vxdctl enable and that also didn’t helped.
3. Tried to stop the cluster locally and then start, which also did not helped. Finally rebooted the server which was also not helpful.
4. Checked the output of “cfgadm -al” and all the fc_fabric were fine.
5. Executed the command “vxdiskconfig” and “devfsadm” and it didn’t helped even.
6. prtvtoc on the SAN disk was showing the following error:
# prtvtoc -s /dev/rdsk/c3t50060E800327D00Bd24s2
prtvtoc: /dev/rdsk/c3t50060E800327D00Bd24s2: Unable to read Disk geometry errno = 0x5
7. Since I/O fencing was used in this server cluster, decided to check IO fencing on the SAN disks.
checked the keys on one of the SAN disks and here is the output:
# /sbin/vxfenadm -G /dev/rdsk/c3t50060E800327D00Bd25s2
Reading SCSI Registration Keys…
Device Name: /dev/rdsk/c3t50060E800327D00Bd25s2
Total Number Of Keys: 2
key[0]:
Node ID: 0 Node Name: server1
Key Value: PGR0003
key[1]:
Node ID: 0 Node Name: server1
Key Value: PGR0003
As you can see here, there was only one system name in the output, which means keys were not being shared by the other node (server2). Tried to stop FEN and then start again, but that didn’t helped.
SOLUTION:
—————–

The following solution was adopted:
a. Stop the cluster on both the nodes
# hastop -all
b. Stop FEN on both the nodes
# /etc/init.d/vxfen stop
c. Clear the keys on correct node (server1 in my case).
# /opt/VRTSvcs/vxfen/bin/vxfenclearpre
d. Do the same on other node also (server2)
# /opt/VRTSvcs/vxfen/bin/vxfenclearpre
e. Perform the following commands on node 2 (the one which is faulty, server2)
# devfsadm -C
# vxdctl enable
# vxdctl scandisk
# prtvtoc /dev/rdsk/c3t50060E800327D00Bd25s2
Similarly, do the prtvtoc for all other SAN disks which were not visible.
f. Reboot both the nodes
g. After this, I could see that both the nodes were able to see and use the SAN disks. The following was the successful output of the vxfenadm command:
# /sbin/vxfenadm -G /dev/rdsk/c3t50060E800327D00Bd25s2
Reading SCSI Registration Keys…
Device Name: /dev/rdsk/c3t50060E800327D00Bd25s2
Total Number Of Keys: 4
key[0]:
Node ID: 0 Node Name: server1
Key Value: PGR0003
key[1]:
Node ID: 1 Node Name: server2
Key Value: PGR0003
key[2]:
Node ID: 0 Node Name: server1
Key Value: PGR0003
key[3]:
Node ID: 1 Node Name: server2
Key Value: PGR0003

LEAVE A REPLY

Please enter your comment!
Please enter your name here