Hitachi

For Linux(R) (x86) Systems HA Monitor Cluster Software


6.12.1 Configuring a shared disk

Shared disk configurations differ depending on the handover method of the shared disk: by using LVM, by using HA Booster, or by switching the file system directly created in a disk or partition for which LVM is not used. The following table provides an overview of the shared disk configurations. Note that, in the following table, connecting volume groups for referencing and updating has the same meaning as activating volume groups.

Table 6‒7: Overview of shared disk configurations

Required setting

Shared disk switching method

Remarks

LVM

HA Booster

Disk or partition that does not use LVM

Configuring volume groups

Creating volume groups

Create the volume groups to be switched.

Create the volume groups on the device that is to be registered in the control group.#1

No specification needed.

See (1) Configuring volume groups.

Editing the /etc/rc.sysinit file (or the /etc/lvm/lvm.conf file#2).

Set the volume groups that are not to be connected automatically for referencing and updating when the system starts.

Set the volume groups that are to be connected automatically for referencing and updating when the system starts.#1

No specification needed.

Specifying the HA Booster control group

No specification needed.

Specify settings.

No specification needed.

--

Specifying file systems

Specify settings.

Specify settings.

Specify settings.

Required if file systems are to be switched.

See (2) Specifying file systems.

Specifying character special files

Create an HA Monitor's user command to bind a character special file at the time of system startup and unbind at the time of server shutdown.

Bound at the time of system startup.

HA Monitor does not control character special files (raw devices). To use character special files, consider using another means such as user commands.

Required if character special files are to be switched.

See (3) Specifying character special files.

#1

Applicable when LVM is also used.

#2

If the OS is RHEL7 or a later version, edit the /etc/lvm/lvm.conf file.

Organization of this subsection

(1) Configuring volume groups

(a) Creating volume groups

Creating volume groups that are to be switched by LVM

The system administrator uses an OS command to create volume groups because switchover occurs on the shared disk in units of volume groups.

Note the following about creating volume groups:

  • To create a hot-standby switchover configuration with different servers, configure shared disks so that the data used by each server belongs to a different volume group.

    If a resource server is used for a server group, you can place the data to be used by all the servers in the server group in the same volume group.

  • For a volume group created on a shared disk, assign the same name on both hosts.

  • HA Monitor controls the volume groups created on the shared disk. Therefore, the volume groups must be set to be not connected automatically for referencing and updating when the system starts. If the OS is RHEL7 or a later version, edit the /etc/lvm/lvm.conf file. If the OS is RHEL6, edit the /etc/rc.sysinit file.

(b) Editing the /etc/rc.sysinit file

This subsection explains editing of the /etc/rc.sysinit file.

  • When using LVM for switchover processing

    Search the file for the lines that contain "/sbin/lvm vgchange -a y" or "/sbin/lvm vgchange -a ay", and then delete "/sbin/lvm.static vgchange -a y --ignorelockingfailure".

    If a command to be executed by action is deleted as a result of the above deletion, either delete action itself or comment it out. If deletion of action results in deletion of a command that is executed by the then clause of the if command, insert a null line by using a colon (:) to avoid shell syntax errors.

    If the volume group contains a resource other than the shared disk to which you want to connect automatically for referencing and updating at system startup, add the following information immediately after the deleted line: "/sbin/lvm.static vgchange -a y name-of-volume-group-to-be-connected --sysinit" or "/sbin/lvm vgchange -a ay name-of-volume-group-to-be-connected --sysinit".

Important

Use special caution when editing the /etc/rc.sysinit file. If you edit it incorrectly, the system might be unable to start. For example, the system becomes unable to start in the following cases:

  • The /etc/rc.sysinit file is incorrectly edited and a shell syntax error occurs.

  • LVM is allocated in /root, /home, or another area that is required for startup and connection by the vgchange -a y command is not specified in the /etc/rc.sysinit file.

After editing the /etc/rc.sysinit file, if you update the initscripts package of Red Hat Enterprise Linux, the /etc/rc.sysinit file might be overwritten. If the file is overwritten, the changes to the file are lost. If the changes to the /etc/rc.sysinit file are lost, you must edit it again.

The following subsection presents examples of editing the /etc/rc.sysinit file.

(c) Examples of editing the /etc/rc.sysinit file (when using LVM)

This subsection presents examples of editing the /etc/rc.sysinit file when using LVM.

The following show an example when the OS being used is Red Hat Enterprise Linux 6.1.

Before editing (when all volume groups in the system are to be connected automatically for referencing and updating)
.
.
if [ -x /sbin/lvm ]; then
    action $"Setting up Logical Volume Management:" /sbin/lvm vgchange -a y --sysinit
fi
.
.
After editing (when the volume group is not to be connected automatically for referencing and updating)
.
.
  if [ -x /sbin/lvm ]; then
#     action $"Setting up Logical Volume Management:" /sbin/lvm vgchange -a y --sysinit
    :
  fi
.
.
After editing (when vg01 and vg02 are to be connected automatically for referencing and updating)
.
.
if [ -x /sbin/lvm ]; then
#    action $"Setting up Logical Volume Management:" /sbin/lvm vgchange -a y --sysinit
     /sbin/lvm vgchange -a y vg01 --sysinit
     /sbin/lvm vgchange -a y vg02 --sysinit
    :
fi
.
.

The following shows an example when the OS being used is Red Hat Enterprise Linux 6.4.

Before editing (when all volume groups in the system are to be connected automatically for referencing and updating)
.
.
if [ -x /sbin/lvm ]; then
    action $"Setting up Logical Volume Management:" /sbin/lvm vgchange -a ay --sysinit
fi
.
.
After editing (when not connecting the volume groups automatically for referencing and updating)
.
.
if [ -x /sbin/lvm ]; then
#     action $"Setting up Logical Volume Management:" /sbin/lvm vgchange -a ay --sysinit
     :
fi
.
.
After editing (when vg01 and vg02 are to be connected automatically for referencing and updating)
.
.
if [ -x /sbin/lvm ]; then
#    action $"Setting up Logical Volume Management:" /sbin/lvm vgchange -a ay --sysinit
     /sbin/lvm vgchange -a ay vg01 --sysinit
     /sbin/lvm vgchange -a ay vg02 --sysinit
fi
.
.

The following shows an example of when the OS being used is Red Hat Enterprise Linux 6.6, 6.7, 6.8, 6.9, or 6.10.

Before editing (when all volume groups in the system are to be connected automatically for referencing and updating)
.
.
if [ -x /sbin/lvm ]; then
    if [ ! -f /.nolvm ] && ! strstr "$cmdline" nolvm ; then
        action $"Setting up Logical Volume Management:" /sbin/lvm vgchange -a ay --sysinit --ignoreskippedcluster
    else
        echo $"Logical Volume Management disabled at boot."
    fi
fi
.
.
After editing (when the volume group is not to be connected automatically for referencing and updating)
.
.
if [ -x /sbin/lvm ]; then
    if [ ! -f /.nolvm ] && ! strstr "$cmdline" nolvm ; then
#        action $"Setting up Logical Volume Management:" /sbin/lvm vgchange -a ay --sysinit --ignoreskippedcluster
   :
    else
        echo $"Logical Volume Management disabled at boot."
    fi
fi
.
.
After editing (when vg01 and vg02 are to be connected automatically for referencing and updating)
.
.
if [ -x /sbin/lvm ]; then
    if [ ! -f /.nolvm ] && ! strstr "$cmdline" nolvm ; then
#        action $"Setting up Logical Volume Management:" /sbin/lvm vgchange -a ay --sysinit --ignoreskippedcluster
         /sbin/lvm vgchange -a ay vg01 --sysinit --ignoreskippedcluster
         /sbin/lvm vgchange -a ay vg02 --sysinit --ignoreskippedcluster
    else
        echo $"Logical Volume Management disabled at boot."
    fi
fi
.
.

(d) Editing the /etc/lvm/lvm.conf file

Edit the "auto_activation_volume_list" entry in the /etc/lvm/lvm.conf file so that the shared disks controlled by HA Monitor are controlled as follows:

  • Volume groups controlled by HA Booster (volume groups on a device registered in a control group):

    The volume groups are automatically connected for referencing and updating (that is, activated) when the system starts.

  • Volume groups not controlled by HA Booster:

    The volume groups are not automatically connected for referencing and updating (that is, activated) when the system starts.

(e) Examples of editing the /etc/lvm/lvm.conf file

This subsection presents examples of editing the /etc/lvm/lvm.conf file when using LVM.

The following shows an example when the OS being used is Red Hat Enterprise Linux Server 7.1.

Before editing (when all volume groups in the system are to be connected automatically for referencing and updating)
activation {
...
...
# If auto_activation_volume_list is defined but empty, no volumes will be
# activated automatically and --activate ay/-a ay will do nothing.
#
# auto_activation_volume_list = []
 
# If auto_activation_volume_list is defined and it's not empty, only matching
# volumes will be activated either automatically or by using --activate ay/-a ay.
#
# "vgname" and "vgname/lvname" are matched exactly.
# "@tag" matches any tag set in the LV or VG.
# "@*" matches if any tag defined on the host is also set in the LV or VG
#
# auto_activation_volume_list = [ "vg1", "vg2/lvol1", "@tag1", "@*" ]
...
...
}
After editing (when no volume groups are to be connected automatically for referencing and updating)
activation {
...
...
# If auto_activation_volume_list is defined but empty, no volumes will be
# activated automatically and --activate ay/-a ay will do nothing.
#
# auto_activation_volume_list = []
 
# If auto_activation_volume_list is defined and it's not empty, only matching
# volumes will be activated either automatically or by using --activate ay/-a ay.
#
# "vgname" and "vgname/lvname" are matched exactly.
# "@tag" matches any tag set in the LV or VG.
# "@*" matches if any tag defined on the host is also set in the LV or VG
#
# auto_activation_volume_list = [ "vg1", "vg2/lvol1", "@tag1", "@*" ]
 
auto_activation_volume_list = []
...
...
}
After editing (when vg01 and vg02 are to be connected automatically for referencing and updating)
activation {
...
...
# If auto_activation_volume_list is defined but empty, no volumes will be
# activated automatically and --activate ay/-a ay will do nothing.
#
# auto_activation_volume_list = []
 
# If auto_activation_volume_list is defined and it's not empty, only matching
# volumes will be activated either automatically or by using --activate ay/-a ay.
#
# "vgname" and "vgname/lvname" are matched exactly.
# "@tag" matches any tag set in the LV or VG.
# "@*" matches if any tag defined on the host is also set in the LV or VG
#
# auto_activation_volume_list = [ "vg1", "vg2/lvol1", "@tag1", "@*" ]
 
auto_activation_volume_list = [ "vg01", "vg02" ]
...
...
}

The following shows an example when the OS being used is Red Hat Enterprise Linux Server 7.2.

Before editing (when all volume groups in the system are to be connected automatically for referencing and updating)
# Configuration section activation.
activation {
...
...
        # Configuration option activation/auto_activation_volume_list.
        # Only LVs selected by this list are auto-activated.
...
...
        # Example
        # volume_list = [ "vg1", "vg2/lvol1", "@tag1", "@*" ]
        # 
        # This configuration option does not have a default value defined.
...
...
}
After editing (when no volume groups are to be connected automatically for referencing and updating)
# Configuration section activation.
activation {
...
...
        # Configuration option activation/auto_activation_volume_list.
        # Only LVs selected by this list are auto-activated.
...
...
        # Example
        # volume_list = [ "vg1", "vg2/lvol1", "@tag1", "@*" ]
        # 
        # This configuration option does not have a default value defined.

    auto_activation_volume_list = []
...
...
}
After editing (when vg01 and vg02 are to be connected automatically for referencing and updating)
# Configuration section activation.
activation {
...
...
        # Configuration option activation/auto_activation_volume_list.
        # Only LVs selected by this list are auto-activated.
...
...
        # Example
        # volume_list = [ "vg1", "vg2/lvol1", "@tag1", "@*" ]
        # 
        # This configuration option does not have a default value defined.

    auto_activation_volume_list = [ "vg01", "vg02" ]
...
...
}

The following shows an example when the OS is Red Hat Enterprise Linux Server version 7.3 to 7.7 or 8.1 to 8.5.

Before editing (when all volume groups in the system are to be connected automatically for referencing and updating)
# Configuration section activation.
activation {
...
...
# Configuration option activation/auto_activation_volume_list.
...
...
# Example
# auto_activation_volume_list = [ "vg1", "vg2/lvol1", "@tag1", "@*" ]
#
# This configuration option does not have a default value defined.
...
...
}
After editing (when no volume groups are to be connected automatically for referencing and updating)
# Configuration section activation.
activation {
...
...
# Configuration option activation/auto_activation_volume_list.
...
...
# Example
# auto_activation_volume_list = [ "vg1", "vg2/lvol1", "@tag1", "@*" ]
#
# This configuration option does not have a default value defined.
auto_activation_volume_list = []
...
...
}
After editing (when vg01 and vg02 are to be connected automatically for referencing and updating)
# Configuration section activation.
activation {
...
...
# Configuration option activation/auto_activation_volume_list.
...
...
# Example
# auto_activation_volume_list = [ "vg1", "vg2/lvol1", "@tag1", "@*" ]
#
# This configuration option does not have a default value defined.
auto_activation_volume_list = [ "vg01", "vg02" ]
...
...
}

(2) Specifying file systems

If you use file systems on the shared disk, you must specify the file systems for the OS.

HA Monitor controls switching of file systems by mounting and unmounting them. Therefore, you must set the file systems to not be mounted automatically at the time of system startup. For details about the specification method, see the OS documentation.

Note the following points when you specify a file system to be controlled by HA Monitor in the /etc/fstab file:

(3) Specifying character special files

This subsection describes the settings for character special files if LVM or HA Booster is used.

For character special files, the necessity and timing of binding and unbinding depend on the condition of using them.

The following table describes the conditions for using character special files and the necessity of binding and unbinding.

Table 6‒8: Conditions for using character special files and necessity of binding and unbinding

Condition for using character special file

Necessity of binding and unbinding

On a shared disk that is not in a control group managed by HA Booster

A character special file is bound not at system startup, but when a volume group is connected for referencing and updating. It is unbound when the volume group is disconnected. An HA Monitor user command is used to specify the settings.

Because a character special file must be unbound before the volume group is disconnected, you must specify reverse in the deviceoff_order operand# in the HA Monitor environment settings and the server environment definition.

Specify the settings for all servers that use the character special file.

On a shared disk that is in a control group managed by HA Booster

A character special file is bound by using one of the following items at system startup. There is no need to unbind it.

  • /etc/sysconfig/rawdevices

  • A udev rules file under /etc/udev/rules.d

  • HA Booster startup script

#

For details about the deviceoff_order operand in the HA Monitor environment settings, see 8.3.1 HA Monitor environment settings (sysdef). For details about the deviceoff_order operand in the server environment definition, see 8.4.1 Server environment definition (servers).

The subsections below explain the user commands related to specification of character special files. For details about how to create user commands, see 6.19 Creating user commands.

(a) Timing of issuance of a user command

Specify a user command in such a manner that the raw OS command is issued at the following times:

Binding a character special file

  • When the active server starts (-s start) (server type: online)

  • When hot standby processing starts on a standby server

    • When a server failure occurs on the active server (-a start) (server type: standby)

    • When a host failure occurs in the active system (-h start) (server type: standby)

    • When planned hot standby processing is executed from the active system to a standby system (-w start) (server type: standby)

Unbinding a character special file

  • When the active server terminates

    • When a server failure occurs on the active server (-a start) (server type: online)#

    • When planned hot standby processing is executed from the active system to a standby system (-w start) (server type: online)

    • When a server failure occurs on the active server (hot standby is disabled) (-o start) (server type: online)

    • When the active server terminates normally (-e start) (server type: online)

    • When the active server is terminated by planned termination (-p start) (server type: online)#

  • When the standby server starts (-s start) (server type: standby)

  • When hot standby processing fails on a standby server (-f start) (server type: standby)

#

If the server is in the monitor mode or it is a resource server, no parameters are passed. Therefore, unbinding is not necessary at this timing.

(b) Coding example for a user command

This coding example for a user command binds a character special file (/dev/raw/raw1) to a logical volume (lvol0) on a shared disk (/dev/vg10) for a server (LinuxTP1).

This code is stored under the file name usrcommand_raw_bind in HA Monitor's sample file directory.

#! /bin/sh
 
# The definition of server
SERV="LinuxTP1"                  # server name:LinuxTP1
# user name
USER_NAME="TP1user"              # user name, TPI administrator, or HiRDB administrator
# The definition of device
RAW_NAME="/dev/raw/raw1"         # raw device name
DEV_NAME="/dev/vg10/lvol0"       # the name of logical volume on shared disk
 
# Unbinding the raw device
func_unbind(){
# Wait to unbind the raw device.
COUNT=0
while [ \( ${COUNT} -lt 10 \) -a \( -e "$1" \) ]
do
   # Kill processes accessing the raw device.
   fuser -k "$1" >/dev/null 2>&1
   # Wait for 1 second until processing finishes.
   sleep 1
   # Unbind the raw device.
   raw "$1" 0 0
   COUNT=`expr ${COUNT} + 1`
done
}
 
# Binding the raw device
func_bind(){
# Bind the raw device.
raw "$1" "$2"
# Wait to bind the raw device.
COUNT=0
while [ ${COUNT} -lt 10 ]
do
   # If the raw device can be read, finish binding.
   dd of=/dev/null if="$1" bs=1 count=0 >/dev/null 2>&1
   if [ "$?" = 0 ]
   then
     break
   fi
   # Wait for 1 second until the raw device sets.
   sleep 1
   COUNT=`expr ${COUNT} + 1`
done
# Change the owner of the device
chown $USER_NAME "$1"
}
# Main
# Bind the raw device, when the online server starts or Hotstandby is started.
# Unbind the raw device, when the online server ends, standby server starts
# or Hotstandby is failed.
PATH=/bin:/usr/bin:/sbin
export PATH
if [ "$2" = "${SERV}" ]
then
   if [ "$4" = "online" ]
   then
     case "$5" in
     "-s" )                      # online server startup
       if [ "$6" = "start" ]
       then
         func_bind ${RAW_NAME} ${DEV_NAME}
       fi
     ;;
     "-a" | "-w" | "-o" | "-e" | "-p" )  # online server end
                                 # (server error | planned system switch
                                 # | server error (no standby server)
                                 # | normal end | planned stop)
       if [ "$6" = "start" ]
       then
         func_unbind ${RAW_NAME} ${DEV_NAME}
       fi
     ;;
     esac
   else
     case "$5" in
     "-a" | "-h" | "-w" )        # Standby server is done into Hotstandby.
                                 # (server error | host down | planned system switch)
       if [ "$6" = "start" ]
       then
         func_bind ${RAW_NAME} ${DEV_NAME}
       fi
     ;;
     "-s" | "-f" )               # standby server startup or Hotstandby fail
       if [ "$6" = "start" ]
       then
         func_unbind ${RAW_NAME} ${DEV_NAME}
       fi
     ;;
     esac
   fi
fi

This coding example binds a character special file by executing func_bind in the active system when the active server starts or on the target host when hot standby processing is performed. It also unbinds the character special file by executing func_unbind in the active system when the active server terminates, in a standby system when the standby server starts, or in a standby system resulting in a hot standby error in the event that hot standby processing fails.

func_bind uses the raw command to bind a character special file. After that, func_bind retries the following processing once a second (for a maximum of 10 seconds) until the character special file becomes accessible:

  • dd command

    Checks whether the character special file has become accessible.

  • sleep command

    Waits until the character special file is bound.

  • chown command

    Changes the owner of the character special file. Note that if the OS is RHEL6 or later, you do not need to use a user command to change the owner. You can change the owner by using the udev function of the OS. For details about the udev settings, see the documentation for the OS.

In addition, the owner of the bound character special file is set to the specified user.

func_unbind retries the following processing once a second (for a maximum of 10 seconds) until the character special file is unbound:

  • fuser command

    If a process is using the character special file during hot standby processing, disconnection of the shared disk fails because the character special file cannot be unbound. Therefore, the fuser command is used to forcibly terminate all processes that are using the character special file.

  • sleep command

    Waits until the character special file is unbound.

  • raw command

    Unbinds the character special file.

Note that in this coding example, if bind processing takes 10 seconds or longer for any reason, the chown command, which is executed last, fails to change the privilege. If the settings are specified so that server startup processing continues#, the user cannot notice the error until the server accesses a raw device.

#

uoc_neck is omitted or nouse is specified in the server environment settings.

To prevent this, consider the following measures:

  • In the server environment settings, specify use for uoc_neck so that server startup processing stops when the chown command fails to change the privilege.

    In the sample user command, the result of the chown command, which is executed last, is the result of the user command. However, if you change the sample processing (for example, processing is added after the chown command), make sure that the results of the chown command and other processing are reflected to the final result of the user command.

  • If you do not set use for uoc_neck in the server environment settings, add processing that outputs a message to syslog or another location to the user command as a means for error notification.

  • If necessary, change the interval at which processing is retried until the character special file becomes accessible.