Linux下三种配置iscsi的方法

2022/08/24 posted in  iscsi服务相关

linux下面配置iscsi有多种方法,用户态的有tgt,内核态的有scst和lio,如果只是普通的使用来说,没有太大的区别,对于某些特殊的场景就存在区别了

比如tgt不支持pacemaker里面的 Persistent (SCSI-3) Reservations,而scst支持
tgt不支持vmware的vaai的某些属性,而lio支持
比如scst还支持srpt的功能,而tgt也不支持

所以在某些场景下,我们需要根据自己的需要来选择服务,总体上来说iscsi算是一个中间的组件,并不影响数据本身,只是一个对外的服务,满足需求即可

tgt的配置

因为tgt是支持librbd的,也是支持krbd的,所以这个地方存在两种配置,默认的tgt是不支持rbd的,需要自己重新打包支持开启

[root@lab103 ~]# tgtadm --lld iscsi --mode system --op show | grep rbd

打包的文档见这里:打包一个支持rbd的tgt

安装打好的包

[root@lab103 ~]# rpm -ivh scsi-target-utils-*
Preparing...                          ################################# [100%]
Updating / installing...
   1:scsi-target-utils-1.0.84-4.el7   ################################# [ 50%]
   2:scsi-target-utils-rbd-1.0.84-4.el################################# [100%]

检查librdb的支持情况

[root@lab103 ~]# tgtadm --lld iscsi --mode system --op show | grep rbd
    rbd (bsoflags sync:direct)

支持librbd的配置方式

修改配置文件

[root@lab103 ~]# cat /etc/tgt/conf.d/sample.conf
<target iqn.2022-01.com.example:server.target1>
    bs-type rbd
    backing-store rbd/testrbd
</target>

启动服务

[root@lab103 ~]# systemctl start tgtd

查看生成的服务

[root@lab103 ~]# tgt-admin -s
Target 1: iqn.2022-01.com.example:server.target1
    System information:
        Driver: iscsi
        State: ready
    I_T nexus information:
    LUN information:
        LUN: 0
            Type: controller
            SCSI ID: IET     00010000
            SCSI SN: beaf10
            Size: 0 MB, Block size: 1
            Online: Yes
            Removable media: No
            Prevent removal: No
            Readonly: No
            SWP: No
            Thin-provisioning: No
            Backing store type: null
            Backing store path: None
            Backing store flags:
        LUN: 1
            Type: disk
            SCSI ID: IET     00010001
            SCSI SN: beaf11
            Size: 32212 MB, Block size: 512
            Online: Yes
            Removable media: No
            Prevent removal: No
            Readonly: No
            SWP: No
            Thin-provisioning: No
            Backing store type: rbd
            Backing store path: rbd/testrbd
            Backing store flags:
    Account information:
    ACL information:
        ALL

可以看到这里多了个

Backing store type: rbd
Backing store path: rbd/testrbd

客户端登录后查看磁盘

[root@lab101 ~]# parted -s /dev/sdc print
Error: /dev/sdc: unrecognised disk label
Model: IET VIRTUAL-DISK (scsi)
Disk /dev/sdc: 32.2GB
Sector size (logical/physical): 512B/4194304B
Partition Table: unknown
Disk Flags:

可以看到物理扇区的大小为4194304B,这个在很多环境,rac,vmware都可能不识别,需要修改扇区的大小,这个是在tgt的配置文件里面控制的

修改配置文件

[root@lab103 ~]# cat /etc/tgt/conf.d/sample.conf
<target iqn.2022-01.com.example:server.target1>
    bs-type rbd
    backing-store rbd/testrbd
    lbppbe 0
</target>

这个lbppbe 0就是关键配置,设置以后客户端的连接后看到的磁盘扇区就是512b的了

[root@lab101 ~]# parted -s /dev/sdc print
Error: /dev/sdc: unrecognised disk label
Model: IET VIRTUAL-DISK (scsi)
Disk /dev/sdc: 32.2GB
Sector size (logical/physical): 512B/512B
Partition Table: unknown
Disk Flags:

支持krbd的配置方式

修改配置文件

<target iqn.2022-01.com.example:server.target1>
    backing-store /dev/rbd/rbd/testrbd
</target>

启动服务

[root@lab103 ~]# systemctl start tgtd

查看生成的服务

[root@lab103 ~]# tgt-admin -s
Target 1: iqn.2022-01.com.example:server.target1
    System information:
        Driver: iscsi
        State: ready
    I_T nexus information:
    LUN information:
        LUN: 0
            Type: controller
            SCSI ID: IET     00010000
            SCSI SN: beaf10
            Size: 0 MB, Block size: 1
            Online: Yes
            Removable media: No
            Prevent removal: No
            Readonly: No
            SWP: No
            Thin-provisioning: No
            Backing store type: null
            Backing store path: None
            Backing store flags:
        LUN: 1
            Type: disk
            SCSI ID: IET     00010001
            SCSI SN: beaf11
            Size: 32212 MB, Block size: 512
            Online: Yes
            Removable media: No
            Prevent removal: No
            Readonly: No
            SWP: No
            Thin-provisioning: No
            Backing store type: rdwr
            Backing store path: /dev/rbd/rbd/testrbd
            Backing store flags:
    Account information:
    ACL information:
        ALL

这个地方注意下看下客户端的映射的盘的扇区大小

[root@lab101 ~]# parted -s /dev/sdc print
Error: /dev/sdc: unrecognised disk label
Model: IET VIRTUAL-DISK (scsi)
Disk /dev/sdc: 32.2GB
Sector size (logical/physical): 512B/512B
Partition Table: unknown
Disk Flags:

LIO配置

使用命令行配置

创建一个block设备

[root@lab103 ~]# targetcli /backstores/block/ create disk01 /dev/rbd/rbd/testrbd
Created block storage object disk01 using /dev/rbd/rbd/testrbd.

查看当前配置

[root@lab103 ~]# targetcli ls /
o- / ............................................................ [...]
  o- backstores ................................................. [...]
  | o- block ..................................... [Storage Objects: 1]
  | | o- disk01  [/dev/rbd/rbd/testrbd (30.0GiB) write-thru deactivated]
  | |   o- alua ...................................... [ALUA Groups: 1]
  | |     o- default_tg_pt_gp .......... [ALUA state: Active/optimized]
  | o- fileio .................................... [Storage Objects: 0]
  | o- pscsi ..................................... [Storage Objects: 0]
  | o- ramdisk ................................... [Storage Objects: 0]
  o- iscsi ............................................... [Targets: 0]
  o- loopback ............................................ [Targets: 0]

创建一个target

[root@lab103 ~]# targetcli /iscsi/ create iqn.2022-01.org.linux-iscsi.lab103.x8664:sn.192f7d06da4a
Created target iqn.2022-01.org.linux-iscsi.lab103.x8664:sn.192f7d06da4a.
Created TPG 1.
Global pref auto_add_default_portal=true
Created default portal listening on all IPs (0.0.0.0), port 3260.

创建一个lun

[root@lab103 ~]# targetcli  /iscsi/iqn.2022-01.org.linux-iscsi.lab103.x8664:sn.192f7d06da4a/tpg1/luns create /backstores/block/disk01
Created LUN 0.

设置target的属性

[root@lab103 ~]# targetcli  /iscsi/iqn.2022-01.org.linux-iscsi.lab103.x8664:sn.192f7d06da4a/tpg1 set attribute authentication=0 demo_mode_write_protect=0 generate_node_acls=1 cache_dynamic_acls=1

保存配置文件

[root@lab103 ~]# targetcli /  saveconfig
Last 10 configs saved in /etc/target/backup/.
Configuration saved to /etc/target/saveconfig.json

启动target

[root@lab103 ~]# systemctl start target

scst的配置

配置fileio模式

[root@lab103 ~]# cat /etc/scst.conf
HANDLER vdisk_fileio {
    DEVICE disk01 {
        filename /dev/rbd/rbd/testrbd
        nv_cache 1
    }
}

TARGET_DRIVER iscsi {
    enabled 1
    TARGET iqn.2022-05.net.vlnb:tgt {
        LUN 0 disk01
        enabled 1
    }
}

配置blockio模式

[root@lab103 ~]# cat /etc/scst.conf
HANDLER vdisk_blockio {
    DEVICE disk01 {
        filename /dev/rbd/rbd/testrbd
        nv_cache 1
    }
}

TARGET_DRIVER iscsi {
    enabled 1
    TARGET iqn.2022-05.net.vlnb:tgt {
        LUN 0 disk01
        enabled 1
    }
}

启动服务

[root@lab103 ~]# systemctl start scst