[PATCH 0/4] [Target_Core_Mod] Add explict/implict ALUA transition for primary/secondary access states

["Nicholas A. Bellinger" <nab@linux-iscsi.org>]

Greetings folks,

This patch series adds complete fabric level support for Asymmetric
Logical Unit Assignment (ALUA) explict (w/ remote SET_TARGET_PORT_GROUPS
in-band method) and implict (w/ local configfs method) with all SPC-4
defined ALUA primary and secondary access states, as well as the full
ALUA logical unit group abstraction to the lio-core-2.6.git tree.

This patch series has been tested on v2.6.31-rc5 x86 KVM using sg_stpg
for the explict transition case from Open-iSCSI Initiator, and using
lio-utils.git CLI ops for the implict case.  The userspace commits to
make it go can be found here:

http://git.kernel.org/?p=linux/storage/lio/lio-utils.git;a=commitdiff;h=481916be1c568fbab505ade534527d60b33bbb8a
http://git.kernel.org/?p=linux/storage/lio/lio-utils.git;a=commitdiff;h=7e3e2134397145ce75aba2cbd1725abe761bb8bd

Here are the patches:

[PATCH 1/4] [Target_Core_Mod/ALUA]: Add support for explict/implict ALUA primary/secondary state transition
[PATCH 2/4] [Target_Core_Mod/ConfigFS]: Add implict ALUA attributes and move tg_pt_gp list
[PATCH 3/4] [LIO-Target]: Add ALUA Active/NonOptimized state check
[PATCH 4/4] [LIO-Target/ConfigFS]: Add attribute for ALUA secondary port access state

Any comments from the SCSI and multipath folks would be apperciated,

Many thanks for your most valuable of time,

--nab

Here is what it looks like in action with Open-iSCSI Initiators using scsi_dh_alua + multipath:

*) A single TCM storage object (iblock/lvm_test0) has two non default ALUA target port groups
created for itself, lio_alua_west and lio_alua_east.  Each of these two ALUA target port groups
has a diSCSI target port/LUN active from different iSCSI target endpoints:

   target:~# tcm_node --listaluatpgs iblock_0/lvm_test0
   \------> lio_alua_west  Target Port Group ID: 1
            Active ALUA Access Type(s): Implict and Explict
            Primary Access State: Active/Optimized
            Primary Access Status: None
            Preferred Bit: 0
            Active/NonOptimized Delay in milliseconds: 100
            \------> TG Port Group Members
                iSCSI/iqn.2003-01.org.linux-iscsi.target.i686:sn.e475ed6fcdd0/tpgt_1/lun_0
   \------> lio_alua_east  Target Port Group ID: 2
            Active ALUA Access Type(s): Implict and Explict
            Primary Access State: Active/Optimized
            Primary Access Status: None
            Preferred Bit: 0
            Active/NonOptimized Delay in milliseconds: 100
            \------> TG Port Group Members
                iSCSI/iqn.2003-01.org.linux-iscsi.target.i686:sn.cff3eedbd2fd/tpgt_1/lun_0
   <SNIP>

*) On the Initiator side, two sessions are created to each iSCSI target endpoint:

   [  877.235778] sd 4:0:0:0: Attached scsi generic sg8 type 0
   [  877.669556] sd 4:0:0:0: alua: supports implicit and explicit TPGS
   [  877.676442] sd 4:0:0:0: alua: port group 01 rel port 02
   [  877.681561] sd 4:0:0:0: alua: port group 01 state A supports TOUSNA
   [  877.726035] sd 4:0:0:0: alua: port group 01 state A supports TOUSNA
   [  877.746897] sd 3:0:0:0: alua: supports implicit and explicit TPGS
   [  877.755445] sd 3:0:0:0: alua: port group 01 rel port 01
   [  877.760968] sd 3:0:0:0: alua: port group 01 state A supports TOUSNA

The two iSCSI Target port/LUNs appear as /dev/sde and /dev/sdi on the initiator under mpath4:

    mpath4 (36001405a97e4ce21c0711de829b000c2) dm-2 LIO-ORG,IBLOCK
    [size=500M][features=0][hwhandler=1 alua]
    \_ round-robin 0 [prio=100][enabled]
     \_ 3:0:0:0 sde 8:64  [active][ready]
     \_ 4:0:0:0 sdi 8:128 [active][ready]

*) Next using explict ALUA from the initiator, change TG_PT_GP_ID=1 (lio_alua_west) to Active/NonOptimized:

   initiator# sg_stpg -vvv -a /dev/sdi --tp 1
   open /dev/sdi with flags=0x802
       set target port groups cdb: a4 0a 00 00 00 00 00 00 00 08 00 00 
       set target port groups parameter list:
           00 00 00 00 01 00 00 01                             
         duration=8 ms

*) Send a REPORT_TARGET_PORT_GROUPS from the initiator to one of the iSCSI LUNs:

   initiator# sg_rtpg --decode /dev/sde
   Report target port groups:
   <SNIP>
     target port group id : 0x2 , Pref=0
       target port group asymmetric access state : 0x00 (active/optimized)
       T_SUP : 1, O_SUP : 1, U_SUP : 1, S_SUP : 1, AN_SUP : 1, AO_SUP : 1
       status code : 0x00 (no status available)
       vendor unique status : 0x00
       target port count : 01
       Relative target port ids:
         0x02
     target port group id : 0x1 , Pref=0
       target port group asymmetric access state : 0x01 (active/non optimized)
       T_SUP : 1, O_SUP : 1, U_SUP : 1, S_SUP : 1, AN_SUP : 1, AO_SUP : 1
       status code : 0x01 (target port asym. state changed by SET TARGET PORT GROUPS command)
       vendor unique status : 0x00
       target port count : 01
       Relative target port ids:
         0x01

*) Then, using implict ALUA via configfs, change lio_alua_east (TG_PT_GP_ID=2) to Standby:

   target:~# tcm_node --setaluastate iblock_0/lvm_test0 lio_alua_east s
   Successfully set primary ALUA access state for TG PT Group: lio_alua_east to Standby

and then check the status again to see the updated Primary Access State and Status for lio_alua_east:

   target:~# tcm_node --listaluatpgs iblock_0/lvm_test0
   \------> lio_alua_west  Target Port Group ID: 1
            Active ALUA Access Type(s): Implict and Explict
            Primary Access State: Active/NonOptimized
            Primary Access Status: Altered by Explict STPG
            Preferred Bit: 0
            Active/NonOptimized Delay in milliseconds: 100
            \------> TG Port Group Members
                iSCSI/iqn.2003-01.org.linux-iscsi.target.i686:sn.e475ed6fcdd0/tpgt_1/lun_0
   \------> lio_alua_east  Target Port Group ID: 2
            Active ALUA Access Type(s): Implict and Explict
            Primary Access State: Standby
            Primary Access Status: Altered by Implict ALUA
            Preferred Bit: 0
            Active/NonOptimized Delay in milliseconds: 100
            \------> TG Port Group Members
                iSCSI/iqn.2003-01.org.linux-iscsi.target.i686:sn.cff3eedbd2fd/tpgt_1/lun_0
   <SNIP>

*) On the initiator side, check multipath status for lio_alua_east (TG_PT_GP_ID=2) going into Standby:

   mpath4 (36001405a97e4ce21c0711de829b000c2) dm-2 LIO-ORG,IBLOCK
   [size=500M][features=0][hwhandler=1 alua]
   \_ round-robin 0 [prio=10][enabled]
    \_ 3:0:0:0 sde 8:64  [active][ready] 
    \_ 4:0:0:0 sdi 8:128 [active][faulty]

*) Using implict ALUA via configfs, set lio_alua_east (TG_PT_GP_ID=2) back to Active/Optimized:

   target:~# tcm_node --setaluastate iblock_0/lvm_test0 lio_alua_east o
   Successfully set primary ALUA access state for TG PT Group: lio_alua_east to Active/Optimized

*) Set the preferred bit for lio_alua_east (TG_PT_GP_ID=2)

   target:~# tcm_node --setaluapref iblock_0/lvm_test0 lio_alua_east
   Successfully enabled PREFERRED bit for TG Pt Group: lio_alua_east

*) Issue REPORT_TARGET_PORT_GROUPS again from the initiator side to see the updated status:

   initiator# sg_rtpg --decode /dev/sde
   Report target port groups:
   <SNIP>
     target port group id : 0x2 , Pref=1
       target port group asymmetric access state : 0x00 (active/optimized)
       T_SUP : 1, O_SUP : 1, U_SUP : 1, S_SUP : 1, AN_SUP : 1, AO_SUP : 1
       status code : 0x02 (target port asym. state changed by implicit lu behaviour)
       vendor unique status : 0x00
       target port count : 01
       Relative target port ids:
         0x02
     target port group id : 0x1 , Pref=0
       target port group asymmetric access state : 0x01 (active/non optimized)
       T_SUP : 1, O_SUP : 1, U_SUP : 1, S_SUP : 1, AN_SUP : 1, AO_SUP : 1
       status code : 0x01 (target port asym. state changed by SET TARGET PORT GROUPS command)
       vendor unique status : 0x00
       target port count : 01
       Relative target port ids:
         0x01

*) And check the multipath status again from the initiator to see the updates status:

   mpath4 (36001405a97e4ce21c0711de829b000c2) dm-2 LIO-ORG,IBLOCK
   [size=500M][features=0][hwhandler=1 alua]
   \_ round-robin 0 [prio=60][enabled]
    \_ 3:0:0:0 sde 8:64  [active][ready]
    \_ 4:0:0:0 sdi 8:128 [active][ready]


*) And finally, the output for the above sequence from dmesg on the target side showing the
   explict/implict ALUA state transitions and the allocated UNIT ATTENTIONs to notify SCSI Initiator
   Ports of the transitions:

   [iSCSI]: Allocated UNIT ATTENTION, mapped LUN: 0, ASC: 0x2a, ASCQ: 0x06
   [iSCSI]: Allocated UNIT ATTENTION, mapped LUN: 0, ASC: 0x2a, ASCQ: 0x06
   [iSCSI]: Allocated UNIT ATTENTION, mapped LUN: 0, ASC: 0x2a, ASCQ: 0x06
   [iSCSI]: Allocated UNIT ATTENTION, mapped LUN: 0, ASC: 0x2a, ASCQ: 0x06
   [iSCSI]: Allocated UNIT ATTENTION, mapped LUN: 0, ASC: 0x2a, ASCQ: 0x06
   [iSCSI]: Allocated UNIT ATTENTION, mapped LUN: 0, ASC: 0x2a, ASCQ: 0x06
   [iSCSI]: Allocated UNIT ATTENTION, mapped LUN: 0, ASC: 0x2a, ASCQ: 0x06
   Successful explict ALUA transition TG PT Group: lio_alua_west ID: 1 from primary access state: Active/Optimized to Active/NonOptimized
   [iSCSI]: Releasing UNIT ATTENTION condition with INTLCK_CTRL: 0, mapped LUN: 0, got CDB: 0x00 reported ASC: 0x2a, ASCQ: 0x06
   [iSCSI]: Allocated UNIT ATTENTION, mapped LUN: 0, ASC: 0x2a, ASCQ: 0x06
   Successful implict ALUA transition TG PT Group: lio_alua_east ID: 2 from primary access state: Active/Optimized to Standby
   [iSCSI]: Releasing UNIT ATTENTION condition with INTLCK_CTRL: 0, mapped LUN: 0, got CDB: 0x00 reported ASC: 0x2a, ASCQ: 0x06
   [iSCSI]: Allocated UNIT ATTENTION, mapped LUN: 0, ASC: 0x2a, ASCQ: 0x06
   Successful implict ALUA transition TG PT Group: lio_alua_east ID: 2 from primary access state: Standby to Active/Optimized
   [iSCSI]: Releasing UNIT ATTENTION condition with INTLCK_CTRL: 0, mapped LUN: 0, got CDB: 0x00 reported ASC: 0x2a, ASCQ: 0x06